Regression prediction | MATLAB implements PSO-DNN particle swarm optimization algorithm to optimize data multi-input single-output regression prediction of deep neural network

Regression prediction | MATLAB implements PSO-DNN particle swarm optimization algorithm to optimize data multi-input single-output regression prediction of deep neural network

List of effects

basic introduction

Regression prediction | MATLAB implements PSO-DNN particle swarm optimization algorithm to optimize deep neural network data multiple input single output regression prediction
MATLAB realizes PSO-DNN particle swarm optimization algorithm optimizes deep neural network data multiple input single output regression prediction (Matlab complete program and data)
Input 7 features and output 1, that is, multiple input and single output; the
operating environment is Matlab2018 and above, just run the main program main, and the rest are function files that do not need to be run. All programs are placed in a folder, and data is a data set; the command
window Output RMSE, MAE, R2, MAPE.

Model description

Particle Swarm Optimization (PSO) is a heuristic optimization algorithm, usually used to solve complex nonlinear optimization problems. The grouped convolutional neural network (Grouped Convolutional Neural Network, GCNN) is a variant of the convolutional neural network, which can group the input data for convolution operations, thereby reducing the amount of calculation and the number of parameters.
For data multiple input single output regression prediction problem, particle swarm optimization algorithm can be used to optimize the structure and parameters of grouped convolutional neural network to improve prediction accuracy. Specific steps are as follows:

• Defining the objective function
  First, an objective function needs to be defined to measure the prediction accuracy of the deep neural network. In general, root mean square error (RMSE) or mean square error (MSE) can be used as the objective function.

• Determine the network structure and parameters
  Next, you need to determine the structure and parameters of the deep neural network. These parameters can be used as the dimension of particles, and each particle represents a combination of network structure and parameters.

• Initialize particle swarm
  Randomly generate a certain number of particles, each particle represents a combination of network structure and parameters. Each particle has a velocity and a position, the velocity indicates its direction and speed of movement, and the position indicates its current position.

• Update particle velocity and position
  Calculate the velocity and position at the next moment based on the current particle position and velocity. The update formula is as follows:

v_i(t+1) = w * v_i(t) + c1 * rand() * (pbest_i - x_i(t)) + c2 * rand() * (gbest - x_i(t)) x_i(t+1)
= x_i(t) + v_i(t+1)
Among them, v_i(t) represents the velocity of particle i at time t, x_i(t) represents the position of particle i at time t, pbest_i represents the best position of particle i in history, gbest represents the best position in the history of the entire particle swarm, w, c1 and c2 are constants, and rand() is a random number uniformly distributed in the range [0,1].

• Calculation of fitness
  According to the current position of the particle, the value of the objective function is calculated as the fitness of the particle. If the current position is better than the historical optimal position, update the historical optimal position.

• Judging the stop condition
  If the preset stop condition is reached (such as reaching the maximum number of iterations or the value of the objective function is small enough), the algorithm is stopped and the network structure and parameters corresponding to the historical optimal position are returned.

• Repeat steps 4-6.
  If the stop condition is not met, repeat steps 4-6 until the stop condition is met.
  The network structure and parameters corresponding to the historical optimal position finally obtained are the grouped convolutional neural networks optimized by the particle swarm optimization algorithm, which can be used for data multi-input single-output regression prediction problems.

programming

• Complete program and data download: background private message PSO-DNN particle swarm algorithm optimization deep neural network data multi-input single-output regression prediction

%%  参数设置
% ----------------------  修改模型参数时需对应修改fical.m中的模型参数  --------------------------
options = trainingOptions('adam', ...           % Adam 梯度下降算法
         'MaxEpochs', 500, ...                  % 最大训练次数 500
         'InitialLearnRate', best_lr, ...       % 初始学习率 best_lr
         'LearnRateSchedule', 'piecewise', ...  % 学习率下降
         'LearnRateDropFactor', 0.5, ...        % 学习率下降因子 0.1
         'LearnRateDropPeriod', 400, ...        % 经过 400 次训练后 学习率为 best_lr * 0.5
         'Shuffle', 'every-epoch', ...          % 每次训练打乱数据集
         'ValidationPatience', Inf, ...         % 关闭验证
         'L2Regularization', best_l2, ...       % 正则化参数
         'Plots', 'training-progress', ...      % 画出曲线
         'Verbose', false);

%%  训练模型
net = trainNetwork(p_train, t_train, layers, options);

%%  仿真验证
t_sim1 = predict(net, p_train);
t_sim2 = predict(net, p_test );

%%  数据反归一化
T_sim1 = mapminmax('reverse', t_sim1, ps_output);
T_sim2 = mapminmax('reverse', t_sim2, ps_output);
T_sim1=double(T_sim1);
T_sim2=double(T_sim2);
%%  均方根误差
error1 = sqrt(sum((T_sim1 - T_train).^2) ./ M);
error2 = sqrt(sum((T_sim2 - T_test ).^2) ./ N);
%% 参数初始化
popsize=pop;              %种群规模
lenchrom=dim;              %变量字串长度
fun = fobj;  %适应度函数
pc=0.7;                  %设置交叉概率
pm=0.3;                  %设置变异概率
if(max(size(ub)) == 1)
   ub = ub.*ones(dim,1);
   lb = lb.*ones(dim,1);  
end
maxgen=Max_iter;   % 进化次数  

function [gbest,g,Convergence_curve]=PSO(N,T,lb,ub,dim,fobj)
%% 定义粒子群算法参数
% N 种群 T 迭代次数 
%% 随机初始化种群
D=dim;                   %粒子维数
c1=1.5;                 %学习因子1
c2=1.5;                 %学习因子2
w=0.8;                  %惯性权重

Xmax=ub;                %位置最大值
Xmin=lb;               %位置最小值
Vmax=ub;                %速度最大值
Vmin=lb;               %速度最小值
%%
%%%%%%%%%%%%%%%%初始化种群个体（限定位置和速度）%%%%%%%%%%%%%%%%

x=rand(N,D).*(Xmax-Xmin)+Xmin;
v=rand(N,D).*(Vmax-Vmin)+Vmin;
%%%%%%%%%%%%%%%%%%初始化个体最优位置和最优值%%%%%%%%%%%%%%%%%%%
p=x;
pbest=ones(N,1);
for i=1:N
    pbest(i)=fobj(x(i,:)); 
end
%%%%%%%%%%%%%%%%%%%初始化全局最优位置和最优值%%%%%%%%%%%%%%%%%%
g=ones(1,D);
gbest=inf;
for i=1:N
    if(pbest(i)<gbest)
        g=p(i,:);
        gbest=pbest(i);
    end
end
%%%%%%%%%%%按照公式依次迭代直到满足精度或者迭代次数%%%%%%%%%%%%%
for i=1:T
    i
    for j=1:N
        %%%%%%%%%%%%%%更新个体最优位置和最优值%%%%%%%%%%%%%%%%%
        if (fobj(x(j,:))) <pbest(j)
            p(j,:)=x(j,:);
            pbest(j)=fobj(x(j,:)); 
        end
        %%%%%%%%%%%%%%%%更新全局最优位置和最优值%%%%%%%%%%%%%%%
        if(pbest(j)<gbest)
            g=p(j,:);
            gbest=pbest(j);
        end
        %%%%%%%%%%%%%%%%%跟新位置和速度值%%%%%%%%%%%%%%%%%%%%%
        v(j,:)=w*v(j,:)+c1*rand*(p(j,:)-x(j,:))...
            +c2*rand*(g-x(j,:));
        x(j,:)=x(j,:)+v(j,:);
        %%%%%%%%%%%%%%%%%%%%边界条件处理%%%%%%%%%%%%%%%%%%%%%%
        if length(Vmax)==1
            for ii=1:D
                if (v(j,ii)>Vmax)  |  (v(j,ii)< Vmin)
                    v(j,ii)=rand * (Vmax-Vmin)+Vmin;
                end
                if (x(j,ii)>Xmax)  |  (x(j,ii)< Xmin)
                    x(j,ii)=rand * (Xmax-Xmin)+Xmin;
                end
            end           
        else
            for ii=1:D
                if (v(j,ii)>Vmax(ii))  |  (v(j,ii)< Vmin(ii))
                    v(j,ii)=rand * (Vmax(ii)-Vmin(ii))+Vmin(ii);
                end
                if (x(j,ii)>Xmax(ii))  |  (x(j,ii)< Xmin(ii))
                    x(j,ii)=rand * (Xmax(ii)-Xmin(ii))+Xmin(ii);
                end
            end
        end
            
    end
    %%%%%%%%%%%%%%%%%%%%记录历代全局最优值%%%%%%%%%%%%%%%%%%%%%
   Convergence_curve(i)=gbest;%记录训练集的适应度值

end

References

[1] https://blog.csdn.net/kjm13182345320/article/details/129215161
[2] https://blog.csdn.net/kjm13182345320/article/details/128105718