PRC curve

Author: Jin Ge Ge
Link : https://www.zhihu.com/question/30643044/answer/48955833
Source: Zhihu

precision recall curve. Like ROC, first look at the smoothness or not (the blue line is obviously better), and see who goes up and who goes down (on the same test set). Generally speaking, the upper one is better than the lower one (the green line is better than the red line). F1 (the calculation formula is omitted) when P and R are close, the larger the line, the line connecting (0, 0) and (1, 1) is generally drawn. The F1 where the line overlaps with the PRC is the largest F1 ( smooth case), F1 at this time is to PRC as AUC is to ROC. A number is easier to tune a model than a line.

The above two indicators are used to judge whether the model is good or bad , and the graph is somewhat inappropriate. . . But sometimes the model is not simply better than the other (such as the blue and green lines in Figure 2), so the selection of the model should be combined with specific usage scenarios.

The following are two scenarios:
1. Earthquake prediction
For earthquake prediction, we hope that the RECALL is very high, that is to say, we want to predict every earthquake. This time we can sacrifice PRECISION. Better to sound 1,000 alarms and predict 10 earthquakes correctly; don't predict 8 times 100 times and miss two.
2. The conviction of the suspect is
based on the principle of not blaming a good person. We hope that the conviction of the suspect is very accurate. Time to spare some criminals sometimes (low recall), but it's worth it.

For the classifier, it is essentially to give a probability. At this time, we choose a CUTOFF point (threshold), which is higher than this point and judged to be negative. Then the choice of this point needs to be selected according to your specific scene. In turn, the scene will determine the standard when training the model. For example, in the first scene, we only look at the PRECISION when RECALL=99.9999% (the earthquake is all in), and other indicators become meaningless.

If you can only choose one indicator, it must be PRC. A model can be seen clearly.