Decision tree is a common machine learning algorithm, and its idea is very simple, similar to our usual process of making choices by making decisions. It is a structure similar to a flowchart, where each internal node represents a test function, that is, a process (action) similar to making a decision, and each leaf node represents a class label, that is, a decision made after calculating all features ( result). Tags and branches represent the connections that lead to the functionality of these tags. The path from root to leaf represents the classification rules, and the classification and regression problems are solved completely by generating decision rules.
(Due to busy work recently, specific content will be gradually written)
1. Mathematical principles
1. Information entropy
2. Conditional entropy
3. Information gain
4. Information gain ratio
5. Gini index
Second, the implementation process
1. Three steps:
Feature selection, decision tree generation, decision tree pruning
3. Classification
1 、 ID3
2 、 C4.5
3、CART
Reference article:
https://mp.weixin.qq.com/s/k_OjObExgsi4DaHMSGUUMA
https://mp.weixin.qq.com/s/yFxysYAx2Fe–11kJ4M3tg
https://mp.weixin.qq.com/s/lP5ZqfhDCd4Tt3IYpQm -Lg
https://mp.weixin.qq.com/s/poI_7bBnoxgIciaIQYB_Iw
https://mp.weixin.qq.com/s/YzNH1DybIlBTcJsacAEQwA
https://mp.weixin.qq.com/s/XAJnl9HggdQ6-Rab9GJV
