Introduction to the principle of AdaBoost algorithm
The core idea of AdaBoost algorithm
The AdaBoost algorithm (Adaptive Boosting) is an effective and practical Boosting algorithm
that sequentially trains weak learners in a highly adaptive manner. AdaBoost adjusts the weight of the data according to the previous
classification effect. The weight of the misclassified sample in the last weak learner will
increase in the next weak learner, and the weight of the correctly classified sample will decrease accordingly, and will be increased in each round of iteration
. Add a new weak learner to the model. Repeat the process of adjusting weights and training weak learners
until the number of misclassifications is lower than the preset value or the number of iterations reaches the specified maximum number of iterations, thereby obtaining a
strong classifier.
It is preset that the AdaBoost algorithm terminates the iteration when the number of misclassifications is 0, that is, when all the classifications are correct
(that is, the error rate is 0), the iteration is stopped, and the error rate is defined as the sum of the sample weights of the misclassified categories.
For example, for 9 samples, The weight of each sample is 1/9. If there are 2 samples divided incorrectly at this time, then
the error rate at this time is 1/9 + 1/9=2/9
Algorithm steps:
Step 1: Set the initial weight of each sample to be equal to 1/9, and divide them into categories to make the error rate the lowest.
Step 2: The number of misclassifications is 3, and the error rate at this time is 1/9+1/9+1/9=1/3; at this time, increase the weight of wrongly classified samples and decrease the weight of correctly classified samples
.
Step 3: According to the new weight, divide the categories one by one to make the error rate the lowest.
Step 4: The number of misclassifications is 2, but the preset value of 0 is not reached, and step 2 is repeated.
Step 5: Repeat Step 3.
Step 6: The error rate has reached the preset value of 0, and the modeling ends.
Case combat
AdaBoost credit card precision marketing model