Original link: https://blog.csdn.net/weixin_42518879/article/details/83959319
Content: common types of evaluation in machine learning, their meanings and calculations (note that this article for binary classifiers!)
1, confusion matrix
True Positive (real, TP): The positive class prediction for the number of positive class
True Negative (true negative, TN): The negative class type predicted number of negative
False Positive (false positive, FP): The negative class prediction of the number of positive type (Type I error)
False Negative (false negative, FN): n is the number of predicted class of a negative type (Type II error)
2, the precise ratio (Precision)
Accuracy rate indicates the proportion of the embodiment is divided into n where n is a real example of the embodiment.
3, recall (the Recall)
Recall instances actually represents all positive examples is predicted to be positive proportional embodiment, is equivalent to the sensitivity (Sensitive)
4, comprehensive evaluation index (F-Measure)
Precision and Recall the case sometimes appear contradictory, in order to consider them, we used indicator is the F-Measure, F value higher prove more effective model.
F-Measure Recall and Precision is weighted harmonic mean.
When the parameter α = 1, is our most common F1.
5, ROC curve and AUC (Area Under Curve)
In the second category, we will typically calculate a probability value for each sample, and then determining the probability values for the sample belongs to the category, then the time required to set a threshold value designated negative class. Set this threshold will directly affect the Precision and Recall, but a smaller impact for AUC, so we have to carry out a more comprehensive evaluation of the model by doing the ROC curve and the calculated value of the AUC.
ROC curve plotting principle is as follows: Suppose we set a total of n test samples, each sample then we will get a probability to each probability threshold value is calculated at this time of "True Positive Rate" and "False Positive Rate "value, to obtain a total value n (n points). Then "True Positive Rate" on the vertical axis, "False Positive Rate" as abscissa, the n data points plotted ROC curve is drawn.
AUC (Area Under Curve) is the area under the ROC curve.
6, accuracy (Accuracy)
Our accuracy rate is the most commonly used evaluation, is to be correctly predicted the proportion of all instances, but when there is an imbalance of data, the accuracy is not very comprehensive evaluation of the quality of model performance.
7, specific calculation
All of the above evaluation index calculated by sklearn.metrics all relevant functions implemented, reference links: evaluation method described in sklearn.metrics