"Explanatory Machine Learning" Notes (3): Interpretability of Decision Trees

《Interpretable machine learning》 Christophm

A decision tree:

1. Model interpretability

It is very simple to explain the decision tree. Starting from the root node, according to the judgment of the edge, go to the next subset until you reach the leaf node and get the result. All edges are connected by AND.
The interpretation template can be: if the feature x is greater than the threshold c [larger/smaller] AND ... then the prediction result is the average value of all instances y in the leaf node.

Measure the importance of features:
Calculation method:
traverse all the division points that use this feature, and calculate how much (proportion) it reduces the variance or Gini index of the result compared with the parent node. On the other hand, it also shows that the importance of each feature can be understood as part of the interpretability of the entire model.

Tree decomposition (single instance explanation):
There is a large section written in the book, and the integration is to restore the path passed by the instance and accumulate the contributions of the passed nodes (features).

2. Examples

Small decision tree for predicting bike rental volume:

The structure of the tree shows that both temperature and time trends are used for segmentation, but does not quantify which feature is more important. So we did a quantitative analysis with variance:

According to the results, the time trend is far more important than the temperature.

3. Advantages and disadvantages

advantage:

1. It is suitable for capturing the interaction information between features.
2. The explanation is very simple.
3. The structure of the tree is directly visualized in the model.
4. The resulting explanations are human-friendly.

shortcoming:

1. cannot handle linear relationships.
2. Unstable, due to the hierarchical nature of the tree, different training sets will lead to completely different decision trees.
3. Not smooth, a small change in the feature value may affect the classification result.
4. As the depth increases, the number of leaf nodes increases dramatically.