sklearn adaboost source code analysis

CSDN

sklearn.ensemble._weight_boosting.BaseWeightBoosting.fit

for iboost in range(self.n_estimators):
    # Boosting step
    sample_weight, estimator_weight, estimator_error = self._boost(
        iboost,
        X, y,
        sample_weight,
        random_state)

    sample_weight_sum = np.sum(sample_weight)

    if iboost < self.n_estimators - 1:
        # Normalize
        sample_weight /= sample_weight_sum

sklearn.ensemble._weight_boosting.AdaBoostClassifier._boost_discrete

estimator.fit(X, y, sample_weight=sample_weight)

sample_weightIs $w_{mi}$, Fitting according to sample weight

# Instances incorrectly classified
incorrect = y_predict != y

# Error fraction
estimator_error = np.mean(
    np.average(incorrect, weights=sample_weight, axis=0))

The misclassification rate is the sum of the weights of the misclassified samples: $e_m={\sum }_{i=1}^N{w}_{mi}I(G_m(x_i)\ne Y_i)$

If the effect of the weak learner is not even random, stop early.

Calculate $G_m$The coefficient of $a_m=\frac{1}{2}log\frac{1-e_m}{e_m}$

SAMMEThe algorithm considers multiple classifications, and multiplies it by the attenuation of learning_rate to achieve regularization.

$w_{mi}=exp(-y_i{a}_{m-1}{G}_{m-1}(x_i))$

试想 ，$Y\times \hat{Y}$, In the second classification, only when the two are different, it is 1

In fact, this writing also takes into account the multi-category situation. For multi-category, you only need to modify $a_m$That is estimator_weightthe calculation in the code .