The key indicators for two-classification problems (precision, recall, accuracy, F1-score) and the key indicators for multi-classification problems (Macro-average, Micro-average, Weighted-average, accuracy) mentioned in the previous article can help us understand: What indicators can be used to roughly evaluate the quality of the classification model.
According to personal understanding, the ROC curve, AUC area and PR curve that will be discussed in this article are to visualize these indicators , allowing different model comparisons and threshold selection .
1. ROC curve
Examples of chestnuts:
The accuracy of this model reached 0.9. In fact, the model did not identify a single positive. It can be seen that accuracy is not a good indicator when used alone.
Several new indicators are introduced at this time:
TPR is Recall recall rate.
The threshold can be adjusted, calculate the FPR and TPR corresponding to each threshold, and draw a line with FPR as the abscissa and TPR as the ordinate, which is the ROC curve.
The best threshold needs to depend on the situation.
2. AUC area
In addition to threshold adjustment, the ROC curve can also be used for model comparison. There are three ROC curves in the figure below.
AUC is the area under the ROC curve. AUC can be used to judge the quality of the model. Generally speaking, the larger the AUC value, the better.
There are three ROC curves in the above figure. If you evaluate these three models, you can evaluate them from two aspects. First, from the perspective of the ROC curve, the closer the model is to the upper left corner, the better, so A>B>C; second, from the perspective of the AUC value, the larger the AUC value, the better, so it is also A>B>C.
3. PR curve
By continuously adjusting the threshold and calculating the Precision and Recall corresponding to each threshold, the resulting curve is the PR curve.