Multidimensional time series | MATLAB implements WOA-CNN-GRU-Attention multi-variable time series prediction (SE attention mechanism)
Table of contents
Prediction effect
Basic description
1. Matlab implements WOA-CNN-GRU whale algorithm to optimize convolution gated cyclic unit multi-variable time series prediction;
2. The running environment is Matlab2021b;
3...data is a data set, excel data, input multiple features, and output a single variable, Considering the influence of historical characteristics, multi-variable time series prediction,
main.m is the main program, just run it, all files are placed in one folder;
4. The command window outputs R2, MSE, MAE, MAPE and MBE multi-index evaluation;
5 .Whale algorithm optimizes learning rate, hidden layer nodes, and regularization coefficients;
Model description
Attention mechanism module:
SEBlock (Squeeze-and-Excitation Block) is a new structural unit focusing on the channel dimension. It adds a channel attention mechanism to the model. This mechanism adds the importance of each feature channel. Weights enhance or suppress corresponding channels for different tasks to extract useful features. The internal operation process of this module is shown in the figure, which is divided into three steps: The first is the Squeeze compression operation, which compresses the features of the spatial dimension while keeping the number of feature channels unchanged. Fusion of global information is global pooling, and each two-dimensional feature channel is converted into a real number. The real number calculation formula is as shown in the formula. The real number is obtained by dividing the sum of the features obtained by k channels by the value of the spatial dimension, which is H*W. The second is the Excitation excitation operation, which consists of two fully connected layers and a Sigmoid function. As shown in the formula, s is the output of the excitation operation, σ is the activation function sigmoid, W2 and W1 are the corresponding parameters of the two fully connected layers respectively, δ is the activation function ReLU, and the feature is first reduced in dimension and then increased in dimension. The last step is the Reweight operation, which weights the previous input features channel by channel to complete the redistribution of the original features on each channel.
programming
- Complete program and data acquisition method: privately message the blogger to reply to MATLAB to implement WOA-CNN-GRU-Attention multi-variable time series prediction (SE attention mechanism) .
%% 优化算法参数设置
SearchAgents_no = 8; % 数量
Max_iteration = 5; % 最大迭代次数
dim = 3; % 优化参数个数
lb = [1e-3,10 1e-4]; % 参数取值下界(学习率，隐藏层节点，正则化系数)
ub = [1e-2, 30,1e-1]; % 参数取值上界(学习率，隐藏层节点，正则化系数)
fitness = @(x)fical(x,num_dim,num_class,p_train,t_train,T_train);
[Best_score,Best_pos,curve]=WOA(SearchAgents_no,Max_iteration,lb ,ub,dim,fitness)
Best_pos(1, 2) = round(Best_pos(1, 2));
best_hd = Best_pos(1, 2); % 最佳隐藏层节点数
best_lr= Best_pos(1, 1);% 最佳初始学习率
best_l2 = Best_pos(1, 3);% 最佳L2正则化系数
%% 建立模型
lgraph = layerGraph(); % 建立空白网络结构
tempLayers = [
sequenceInputLayer([num_dim, 1, 1], "Name", "sequence") % 建立输入层，输入数据结构为[num_dim, 1, 1]
sequenceFoldingLayer("Name", "seqfold")]; % 建立序列折叠层
lgraph = addLayers(lgraph, tempLayers); % 将上述网络结构加入空白结构中
tempLayers = [
convolution2dLayer([3, 1], 16, "Name", "conv_1", "Padding", "same") % 建立卷积层,卷积核大小[3, 1]，16个特征图
reluLayer("Name", "relu_1") % Relu 激活层
lgraph = addLayers(lgraph, tempLayers); % 将上述网络结构加入空白结构中
tempLayers = [
sequenceUnfoldingLayer("Name", "sequnfold") % 建立序列反折叠层
flattenLayer("Name", "flatten") % 网络铺平层
fullyConnectedLayer(num_class, "Name", "fc") % 分类层
lgraph = addLayers(lgraph, tempLayers); % 将上述网络结构加入空白结构中
lgraph = connectLayers(lgraph, "seqfold/out", "conv_1"); % 折叠层输出 连接 卷积层输入
lgraph = connectLayers(lgraph, "seqfold/miniBatchSize", "sequnfold/miniBatchSize");
% 折叠层输出连接反折叠层输入
lgraph = connectLayers(lgraph, "relu_2", "sequnfold/in"); % 激活层输出 连接 反折叠层输入
%% 参数设置
options = trainingOptions('adam', ... % Adam 梯度下降算法
'MaxEpochs', 500,... % 最大训练次数
'InitialLearnRate', best_lr,... % 初始学习率为0.001
'L2Regularization', best_l2,... % L2正则化参数
'LearnRateSchedule', 'piecewise',... % 学习率下降
'LearnRateDropFactor', 0.1,... % 学习率下降因子 0.1
'LearnRateDropPeriod', 400,... % 经过训练后 学习率为 0.001*0.1
'Shuffle', 'every-epoch',... % 每次训练打乱数据集
'ValidationPatience', Inf,... % 关闭验证
'Plots', 'training-progress',... % 画出曲线
'Verbose', false);
%% 训练
net = trainNetwork(p_train, t_train, lgraph, options);
References
[1] https://blog.csdn.net/kjm13182345320/article/details/129036772?spm=1001.2014.3001.5502
[2] https://blog.csdn.net/kjm13182345320/article/details/128690229