Regression prediction | MATLAB implements multi-input single-output regression prediction based on SVM-RFE-BP support vector machine recursive feature elimination feature selection algorithm combined with BP neural network
Table of contents
predictive effect
basic introduction
MATLAB implements SVM-RFE-BP support vector machine recursive feature elimination feature selection algorithm combined with BP neural network multi-input single-output regression prediction, the output is the serial number of
the
selected
feature Type: SVM setting type (default 0)
0 – C-SVC
1 --v-SVC
2 – one-class SVM
3 – e-SVR
4 – v-SVR
-t Kernel function type: Kernel function setting type (default 2)
0 – Linear: u'v
1 – Polynomial: (r u'v + coef0)^degree
2 – RBF function: exp(-r|uv|^2)
3 –sigmoid: tanh(r u'v + coef0)
after feature After selection, the serial number of reserved features is:
126 160 161 163 165 166 237 239 240 370
The evaluation results are as follows:
the average absolute error MAE is: 0.27933 the mean
square error MSE is: 0.15813
the root mean square error RMSEP is: 0.39765
the coefficient of determination R ^2 is: 0.93392
Remaining Prediction Residual RPD is: 4.2631
Mean Absolute Percentage Error MAPE is: 0.0032299
research content
Multi-input single-output regression prediction based on SVM-RFE-BP feature selection algorithm combined with BP neural network is a method that combines support vector machine recursive feature elimination (SVM-RFE) and backpropagation (BP) neural network. The following are the basic steps of the algorithm:
Data preparation: Prepare a training dataset containing multiple input features and an output variable. Make sure the dataset has been preprocessed and normalized.
Feature Selection: The input features are sorted and selected using the SVM-RFE algorithm. SVM-RFE is a recursive feature elimination algorithm that repeatedly trains a support vector machine (SVM) model and removes the least important features until a specified number of features is reached or a certain stopping criterion is reached.
Feature extraction: Using the features selected by SVM-RFE as input, these features are extracted from the training dataset.
Neural Network Construction: Build a BP neural network model with appropriate input, hidden and output layers. The number of nodes in the input layer should be the same as the number of features selected, and the number of nodes in the output layer should be 1.
Neural network training: use the feature extraction data as input and output variable as the target to train the BP neural network. Use the backpropagation algorithm to update the weights and biases of the network to minimize the error between the predicted output and the actual output.
Prediction: Use the trained BP neural network model to predict new input features. Input these features into the trained neural network to get the corresponding output.
This method based on SVM-RFE-BP can combine the feature selection ability of support vector machine and the nonlinear modeling ability of neural network to improve the performance and accuracy of regression prediction. However, it should be noted that the effect of this method depends on the characteristics of the dataset and the parameter settings of feature selection, so proper tuning and validation are required in practical applications.
programming
- Complete source code and data acquisition method 1: Private letter bloggers reply SVM-RFE-BP regression or exchange of programs of equal value;
- Complete program and data download method 2 (download directly from the resource): MATLAB implements multi-input single-output regression prediction based on SVM-RFE-BP support vector machine recursive feature elimination feature selection algorithm combined with BP neural network
- Complete program and data download method 3 (subscribe to the "Smart Learning" column, and get 4 copies of the program included in the "Smart Learning" column, and private message me to get the data after subscription): MATLAB implements recursive feature elimination based on SVM-RFE-BP support vector machine The selection algorithm combined with the multi-input single-output regression prediction of BP neural network can only be obtained outside the column.
%% 输出选择特征的对应序号
disp('经过特征选择后，保留特征的序号为：')
disp(save_index)
%% 特征选择后的数据集
p_train = p_train(:, save_index);
p_test = p_test (:, save_index);
%% 矩阵转置适应模型
p_train = p_train'; p_test = p_test';
t_train = t_train'; t_test = t_test';
%% 创建网络
%% 设置训练参数
net.trainParam.epochs = 1000; % 最大迭代次数
%% 仿真测试
t_sim1 = sim(net, p_train);
t_sim2 = sim(net, p_test );
%% 数据反归一化
T_sim1 = mapminmax('reverse', t_sim1, ps_output);
T_sim2 = mapminmax('reverse', t_sim2, ps_output);
%% 测试集结果
figure;
plotregression(T_test,T_sim2,['回归图']);
figure;
ploterrhist(T_test-T_sim2,['误差直方图']);
%% 均方根误差 RMSE
error1 = sqrt(sum((T_sim1 - T_train).^2)./M);
error2 = sqrt(sum((T_test - T_sim2).^2)./N);
%%
%决定系数
R1 = 1 - norm(T_train - T_sim1)^2 / norm(T_train - mean(T_train))^2;
R2 = 1 - norm(T_test - T_sim2)^2 / norm(T_test - mean(T_test ))^2;
%%
%均方误差 MSE
mse1 = sum((T_sim1 - T_train).^2)./M;
mse2 = sum((T_sim2 - T_test).^2)./N;
%%
%RPD 剩余预测残差
SE1=std(T_sim1-T_train);
RPD1=std(T_train)/SE1;
SE=std(T_sim2-T_test);
RPD2=std(T_test)/SE;
%% 平均绝对误差MAE
MAE1 = mean(abs(T_train - T_sim1));
MAE2 = mean(abs(T_test - T_sim2));
%% 平均绝对百分比误差MAPE
MAPE1 = mean(abs((T_train - T_sim1)./T_train));
MAPE2 = mean(abs((T_test - T_sim2)./T_test));
%% 训练集绘图
figure
%plot(1:M,T_train,'r-*',1:M,T_sim1,'b-o','LineWidth',1)
plot(1:M,T_train,'r-*',1:M,T_sim1,'b-o','LineWidth',1.5)
legend('真实值','SVM-RFE预测值')
xlabel('预测样本')
ylabel('预测结果')
%% 预测集绘图
figure
plot(1:N,T_test,'r-*',1:N,T_sim2,'b-o','LineWidth',1.5)
legend('真实值','SVM-RFE预测值')
xlabel('预测样本')
ylabel('预测结果')
%% 测试集误差图
figure
ERROR3=T_test-T_sim2;
plot(T_test-T_sim2,'b-*','LineWidth',1.5)
%% 绘制线性拟合图
%% 训练集拟合效果图
figure
plot(T_train,T_sim1,'*r');
xlabel('真实值')
ylabel('预测值')
title(string)
hold on ;h=lsline;
set(h,'LineWidth',1,'LineStyle','-','Color',[1 0 1])
%% 预测集拟合效果图
figure
plot(T_test,T_sim2,'ob');
xlabel('真实值')
ylabel('预测值')
string1 = {
'测试集效果图';['R^2_p=' num2str(R2) ' RMSEP='
set(h,'LineWidth',1,'LineStyle','-','Color',[1 0 1])
%% 求平均
R3=(R1+R2)./2;
error3=(error1+error2)./2;
%% 总数据线性预测拟合图
tsim=[T_sim1,T_sim2]';
S=[T_train,T_test]';
figure
plot(S,tsim,'ob');
xlabel('真实值')
ylabel('预测值')
References
[1] https://blog.csdn.net/kjm13182345320/article/details/128577926?spm=1001.2014.3001.5501
[2] https://blog.csdn.net/kjm13182345320/article/details/128573597?spm=1001.2014.3001.5501