How does lightgbm and xgb handle unbalanced data

How does lightgbm and xgb handle unbalanced data

It is conceivable that the sample imbalance behavior of 2 classification problems such as ctr and cvr.
The classification killers such as lightgbm and xgb are also commonly used in 2 classification problems such as ctr and cvr. The question is how do lightgbm and xgb cross the sample imbalance problem?

The answer, both lightgbm and xgb models have two parameters is_unbalance = True / False and scale_pos_weight, which can be solved by choosing one of these two parameters.
Specifically, both is_unbalance and scale_pos_weight are changed from the objective function of the model A large positive sample weight, which indirectly reduces the negative sample weight to solve the sample imbalance problem.
Specifically, when is_unbalance = true, the positive sample weight is automatically set to

$Positive sample weight = \ dfrac {Number of negative samples} {Number of positive samples}$

When setting the scale_pos_weight value,
$Positive sample weight = custom scale \ _pos \ _weight value$ .
If you do n’t believe us, let ’s look at the source code,

Lightgbm

// weight for label
label_weights_[0] = 1.0f;
label_weights_[1] = 1.0f;
// if using unbalance, change the labels weight
if (is_unbalance_ && cnt_positive > 0 && cnt_negative > 0) {
  if (cnt_positive > cnt_negative) {
    label_weights_[1] = 1.0f;
    label_weights_[0] = static_cast<double>(cnt_positive) / cnt_negative;
  } else {
    label_weights_[1] = static_cast<double>(cnt_negative) / cnt_positive;
    label_weights_[0] = 1.0f;
  }
}
label_weights_[1] *= scale_pos_weight_;

It can be seen that lightGBM increases the weight of positive samples by increasing the weight of positive sample labels, ie label_weights_ [1] * = scale_pos_weight_;
multiplying the weight of positive samples label_weights_ [1] by scale_pos_weight_, but not the weight of negative samples To multiply the value
to deal with the problem of sample imbalance.

XGBoost

for (omp_ulong i = 0; i < n - remainder; i += 8) {
  avx::Float8 y(&info.labels_[i]);
  avx::Float8 p = Loss::PredTransform(avx::Float8(&preds_h[i]));
  avx::Float8 w = info.weights_.empty() ? avx::Float8(1.0f)
                                       : avx::Float8(&info.weights_[i]);
  // Adjust weight
  w += y * (scale * w - w);
  avx::Float8 grad = Loss::FirstOrderGradient(p, y);
  avx::Float8 hess = Loss::SecondOrderGradient(p, y);
  avx::StoreGpair(gpair_ptr + i, grad * w, hess * w);
}

In xgb, you can see the formula in the source code:
w + = y * (scale * w-w);
That is, when y = 1, positive sample, w + = (scale * w-w);
w = scale * w.
When y = 0, negative sample, w is still the original value, w + = 0, that is, w is still w.