Recommendation system's ranking algorithm-tree model
It mainly includes decision tree, random forest (RF), GBDT, GBDT+LR and deep forest.
The advantage of the tree model is that it can automatically cross and select features in a supervised manner, and it is also a common method or component in ensemble learning. The commonly used methods of the tree model in the industry include GBDT+LR and Xgboost.
1. Decision Tree
1.1 Decision tree algorithm
Decision tree is the basis of ensemble learning models such as random forest and GBDT.
1.1.1 Decision Tree Model
The decision tree is composed of nodes and directed edges. Nodes are divided into internal nodes and leaf nodes. Internal nodes represent a feature or attribute, and leaf nodes represent a class. The process of using a decision tree to make a decision is to start from the root node, test the corresponding feature attributes in the items to be classified, and select the output branch according to its value until it reaches the leaf node, and use the category stored in the leaf node as the decision result.
The figure below is a decision tree of whether a user buys a computer. Age, student, and reputation indicate characteristics, and whether it indicates category.
The essence of decision tree learning is to summarize a set of classification rules from the training data set, usually using heuristic methods, that is, local optimization. The specific method is that every time a feature is selected, the current optimal feature is selected as the division rule, that is, the local optimal feature. Decision tree learning is usually divided into 3 steps: feature selection, decision tree generation and decision tree pruning.