foreword
When it comes to forests, you have to think of trees, because it is the trees that constitute a huge forest, and the "tree" in this article refers to Decision Tree-----decision tree. Random forest is a combination of decision trees, that is to say, random forest = boosting + decision tree, which is easier to understand. Let’s talk about GBDT. The full name of GBDT is Gradient Boosting Decision Tree, which is gradient boosting decision tree, which is different from random The idea of the forest is similar, but it is slightly more difficult than the random forest. Of course, the effect will be much better than the former. Due to my lack of talent and knowledge, this article will only describe the part of the Random Forest algorithm in detail. As for GBDT, I will give a short paragraph for introduction and guidance. Readers can learn by themselves if they are interested.
Random Forest Algorithm
decision tree
If you want to understand the random forest algorithm, you have to mention a decision tree, what is a decision tree, and how to construct a decision tree. The simple answer is that the classification of data is presented in a tree-like structure, and each sub-branch represents a different classification. , as shown in the following figure:
F0 is the initial value here, Ti is a decision tree, and different problems choose different loss functions and initial values. In Ali, this algorithm is called TreeLink. So the next time I hear about the Treelink algorithm, it refers to the gradient boosting tree algorithm. In fact, I omitted a lot of mathematical derivation process here. In addition, I am not an expert, so I can't explain the mathematical part thoroughly, so there is no Mentioned, I hope to have time to study this knowledge in depth in the future.