Time series prediction | MATLAB implements BO-CNN-GRU Bayesian optimized convolution gated recurrent unit time series prediction
Table of contents
List of effects
basic introduction
Based on Bayesian (bayes) optimized convolutional neural network-gated recurrent unit (CNN-GRU) time series forecasting, BO-CNN-GRU/Bayes-CNN-GRU time series forecasting model. Based on Bayesian algorithm (bayes) to optimize convolutional neural network-gated recurrent unit (CNN-GRU) time series prediction, single-column data set. 1. The optimization parameters are: learning rate, hidden
layer nodes, and regularization parameters.
2. Evaluation indicators include: R2, MAE, MSE, RMSE and MAPE, etc., which are convenient for learning and replacing data.
3. The operating environment is matlab2020b and above.
Model description
-
CNN is built by imitating the mechanism of biological visual perception, capable of supervised learning and unsupervised learning. The sharing of convolution kernel parameters in the hidden layer and the sparsity of inter-layer connections enable CNN to extract deep-level local features from high-dimensional data with a small amount of calculation, and obtain effective representations through convolutional and pooling layers. The structure of the CNN network contains two convolutional layers and a flattening operation, and each convolutional layer contains a convolutional operation and a pooling operation. After the second pooling operation, the fully connected layer is used to flatten the high-dimensional data into one-dimensional data, so that the data can be processed more conveniently.
-
When the number of time steps is large, the historical gradient information of RNN cannot always be maintained within a reasonable range, so gradient decay or explosion is almost inevitable, making it difficult for RNN to capture effective information from long-distance sequences. As a special RNN, LSTM is proposed to solve the problem of gradient disappearance in RNN. GRU is proposed on the basis of LSTM, which has a simpler structure, fewer parameters, shorter training time, and faster training speed than LSTM.
-
In order to make the model have the function of automatically extracting features, the method of deep learning is generally used for construction. Among them, CNN has a strong ability to extract features, and it usually relies on convolution kernels to extract features. However, the existence of convolution kernels limits the long-term dependence of CNNs when processing time series data.
-
In this study, the introduction of GRU can effectively solve this problem, and we can capture the before and after dependencies of time series. On the other hand, the purpose of the GRU module is to capture long-term dependencies, which can learn useful information in historical data for a long time through storage units, and useless information will be forgotten by the forget gate. In addition, directly using the original features for processing will greatly occupy the computing power of the model, thereby reducing the prediction accuracy of the model. The CNN-GRU model combines the advantages of CNN and GRU.
-
Usually, hyperparameters need to be optimized during model training to select an optimal set of hyperparameters for the model to improve the performance and effectiveness of predictions. Setting hyperparameters empirically will make the final model hyperparameter combination not necessarily optimal, which will affect the fitting degree of the model network and its generalization ability to the test data.
-
pseudocode
-
Adjust model parameters by tuning optimization algorithms, learn repetition rate and Bayesian optimization hyperparameters to tune model parameters.
programming
- Complete program and data acquisition method 1: Private message bloggers, program exchange of the same value;
- Complete program and data download method 2 (download directly from the resource): MATLAB implements BO-CNN-GRU Bayesian optimized convolution gated recurrent unit time series prediction
- Complete program and data download method 3 (subscribe to the "Combined Optimization" column, and at the same time get all the programs included in the "Combined Optimization" column, private message me after the data is subscribed): MATLAB implements BO-CNN-GRU Bayesian optimized convolution gating Recurrent Unit Time Series Forecasting
%% 优化算法参数设置
%参数取值上界(学习率，隐藏层节点，正则化系数)
%% 贝叶斯优化参数范围
optimVars = [
optimizableVariable('NumOfUnits', [10, 50], 'Type', 'integer')
optimizableVariable('InitialLearnRate', [1e-3, 1], 'Transform', 'log')
optimizableVariable('L2Regularization', [1e-10, 1e-2], 'Transform', 'log')];
%% 贝叶斯优化网络参数
BayesObject = bayesopt(fitness, optimVars, ... % 优化函数，和参数范围
'MaxTime', Inf, ... % 优化时间（不限制）
'IsObjectiveDeterministic', false, ...
'MaxObjectiveEvaluations', 10, ... % 最大迭代次数
'Verbose', 1, ... % 显示优化过程
'UseParallel', false);
%% 创建混合CNN-GRU网络架构
% 创建"CNN-GRU"模型
layers = [...
% 输入特征
sequenceInputLayer([numFeatures 1 1],'Name','input')
sequenceFoldingLayer('Name','fold')
% CNN特征提取
convolution2dLayer([FiltZise 1],32,'Padding','same','WeightsInitializer','he','Name','conv','DilationFactor',1);
batchNormalizationLayer('Name','bn')
eluLayer('Name','elu')
averagePooling2dLayer(1,'Stride',FiltZise,'Name','pool1')
% 展开层
sequenceUnfoldingLayer('Name','unfold')
% 平滑层
flattenLayer('Name','flatten')
% GRU特征学习
gruLayer(50,'Name','gru1','RecurrentWeightsInitializer','He','InputWeightsInitializer','He')
% GRU输出
gruLayer(NumOfUnits,'OutputMode',"last",'Name','bil4','RecurrentWeightsInitializer','He','InputWeightsInitializer','He')
dropoutLayer(0.25,'Name','drop3')
% 全连接层
fullyConnectedLayer(numResponses,'Name','fc')
regressionLayer('Name','output') ];
layers = layerGraph(layers);
layers = connectLayers(layers,'fold/miniBatchSize','unfold/miniBatchSize');
References
[1] https://blog.csdn.net/kjm13182345320/article/details/129036772?spm=1001.2014.3001.5502
[2] https://blog.csdn.net/kjm13182345320/article/details/128690229