[MATLAB Issue 70] LightGbm (LGBM) gradient boosted decision tree multi-input single-output regression prediction and multi-classification prediction model based on MATLAB (first release on the entire network)
1. Study materials
(LGBM) is an algorithm based on gradient boosted decision tree (GBDT).
The three contents of this study are regression prediction, two-class prediction and multi-class prediction
. Reference link:
lightgbm principle reference link:
training process evaluation index metric function reference link:
lightgbm parameter introduction reference link:
lightgbm parameter adjustment reference link:
2. Regression prediction (multiple inputs and single output)
1. Data settings
data (103 samples, 7 inputs and 1 output)
2. Prediction results
3. Parameter settings
parameters=containers.Map;
parameters('task')='train';
parameters('boosting_type')='gbdt';
parameters('metric')='rmse';
parameters('num_leaves')=31;
parameters('learning_rate')=0.05; %越大，训练集效果越好
parameters('feature_fraction')=0.9;
parameters('bagging_fraction')=0.8;
parameters('bagging_freq')=5;
parameters('num_threads')=1;
parameters('verbose')=1;
4. Training process
[ 1] train rmse 0.208872
[ 2] train rmse 0.203687
[ 3] train rmse 0.202175
[ 4] train rmse 0.200801
[ 5] train rmse 0.199554
[ 6] train rmse 0.196124
[ 7] train rmse 0.193003
[ 8] train rmse 0.192100
[ 9] train rmse 0.189259
[ 10] train rmse 0.186576
............
[ 490] train rmse 0.052932
[ 491] train rmse 0.052870
[ 492] train rmse 0.052847
[ 493] train rmse 0.052830
[ 494] train rmse 0.052820
[ 495] train rmse 0.052771
[ 496] train rmse 0.052689
[ 497] train rmse 0.052619
[ 498] train rmse 0.052562
[ 499] train rmse 0.052506
[ 500] train rmse 0.052457
bestIteration: 500
训练集数据的R2为：0.94018
测试集数据的R2为：0.87118
训练集数据的MAE为：1.365
测试集数据的MAE为：2.3607
训练集数据的MBE为：-0.079848
测试集数据的MBE为：-1.0132
5. Sensitivity analysis of characteristic variables
3. Classification prediction (multiple inputs, single output, two classifications)
1. Data setting
data (357 samples, 12 inputs and 1 output)
2. Prediction results
3. Parameter settings
parameters=containers.Map;
parameters('task')='train';
parameters('boosting_type')='gbdt';
parameters('metric')='binary_error';
parameters('num_leaves')=31;
parameters('learning_rate')=0.05;
parameters('feature_fraction')=0.9;
parameters('bagging_fraction')=0.8;
parameters('bagging_freq')=5;
parameters('num_threads')=1;
parameters('verbose')=0;
4. Training process
[ 0] train binary_error 0.020833
[ 1] train binary_error 0.020833
[ 2] train binary_error 0.020833
[ 3] train binary_error 0.020833
[ 4] train binary_error 0.020833
[ 5] train binary_error 0.020833
[ 6] train binary_error 0.020833
............
[ 191] train binary_error 0.000000
[ 192] train binary_error 0.000000
[ 193] train binary_error 0.000000
[ 194] train binary_error 0.000000
[ 195] train binary_error 0.000000
[ 196] train binary_error 0.000000
[ 197] train binary_error 0.000000
[ 198] train binary_error 0.000000
[ 199] train binary_error 0.000000
bestIteration: 200
5. Sensitivity analysis of characteristic variables
4. Classification prediction (multiple input, single output, multi-classification)
1. Data setting
data (357 samples, 12 inputs and 1 output. 4 categories)
2. Prediction results
3. Parameter settings
parameters=containers.Map;
parameters('task')='train';
parameters('boosting_type')='gbdt';
parameters('metric')='multi_error';
parameters('num_leaves')=31;
parameters('learning_rate')=0.05;
parameters('feature_fraction')=0.9;
parameters('bagging_fraction')=0.8;
parameters('bagging_freq')=5;
parameters('num_threads')=1;
parameters('verbose')=0;
4. Training process
[ 0] train multi_error 0.112500
[ 1] train multi_error 0.066667
[ 2] train multi_error 0.066667
[ 3] train multi_error 0.066667
[ 4] train multi_error 0.062500
[ 5] train multi_error 0.058333
[ 6] train multi_error 0.054167
[ 7] train multi_error 0.054167
[ 8] train multi_error 0.058333
[ 9] train multi_error 0.058333
[ 10] train multi_error 0.054167
[ 11] train multi_error 0.054167
............
[ 190] train multi_error 0.000000
[ 191] train multi_error 0.000000
[ 192] train multi_error 0.000000
[ 193] train multi_error 0.000000
[ 194] train multi_error 0.000000
[ 195] train multi_error 0.000000
[ 196] train multi_error 0.000000
[ 197] train multi_error 0.000000
[ 198] train multi_error 0.000000
[ 199] train multi_error 0.000000
bestIteration: 200
5. Sensitivity analysis of characteristic variables
5. Code acquisition
You can get the download method by replying "Issue 70" to the CSDN backend private message.