Article directory
Basic concepts of classification performance
Similar to the prediction performance evaluation index, the classification performance evaluation index is also to calculate the label value calculated by the model and the actual real label value through the formula of mathematical statistics to obtain the relationship between them, so as to evaluate the model classification Performance is good or bad. In binary classification, common performance classification indicators include accuracy, precision, recall, F1-score, and ROC curve area. In multi-classification, accuracy, precision, recall, and F1-score are mainly used for comparison. At the same time The classification evaluation results can also be displayed using a confusion matrix. The common classification performance evaluation indicators are introduced below.
Before understanding the common classification performance evaluation indicators, you first need to understand the following basic concepts:
Assume that in the classification problem, the class attribute labels are 0 and 1 respectively, and the sample with the class attribute label of 0 is regarded as a positive sample, and the class attribute label A sample with a value of 1 is regarded as a negative sample, and the positive sample that is correctly classified as a positive sample by the model is called a true positive example (True Positive, TP for short), and the negative sample that is correctly classified as a negative sample by the model is called a true negative example ( True Negative, referred to as TN), the negative sample that is misclassified as a positive sample by the model is called a false positive (False Positive, FP for short), and the positive sample that is misclassified as a negative sample by the model is called a false negative sample (False Negative , referred to as FN).
Based on the above concepts, several common classification performance evaluation indicators are as follows:
1. Accuracy
The classification accuracy is mainly obtained by calculating the ratio between the correctly classified data and the total amount of data, and its calculation formula is as follows:
Among them, n_correct represents the number of correctly classified samples, and n_total represents the total number of samples.
1. Accuracy
The precision rate refers to the probability that the actual classification is correct for the samples classified as a certain class attribute label. For the class attribute label 0 and the class attribute label 1, the calculation methods of the precision rate are respectively:
2. Recall rate
The recall rate mainly refers to the probability of classifying a certain class attribute label correctly. Unlike the precision rate, which is based on the classification result, the recall rate is based on the actual result. For the class attribute label 0 and the class attribute label 1, the recall rate are calculated as:
3、F1-score
F1-score is the harmonic mean of precision and recall, and its calculation formula is:
4. ROC curve
In the ROC curve, the horizontal axis represents the false positive rate (False Positive Rate), the vertical axis represents the true positive rate (True Positive Rate), the false positive rate represents the proportion of misclassified negative samples to the total number of negative samples, and the true positive rate represents The proportion of correctly classified positive samples to the total number of positive samples. When the classification effect of the model is better, the curve shifts to the upper left corner of the graph, that is, the area of the ROC curve is used to evaluate the quality of the classification result, and the value range of the area is between 0 and 1.
5. Relevant indicators in multi-classification problems
The biggest difference between multi-classification and binary classification when calculating the above indicators is that multi-classification first needs to be calculated for each different class attribute label, and then the final result is obtained by averaging. There are three commonly used averaging methods at present, which are macro average, micro average and weighted average.
Macro Average (Macro Average) is an arithmetic average of the calculation results of all class attribute labels. Taking the accuracy rate as an example, the calculation formula of the macro average accuracy rate is:
Among them, n represents the total number of categories of class attribute labels, and Precision_i represents the accuracy rate of class attribute label i.
Micro Average (Micro Average) refers to the comprehensive addition calculation of all class attribute label predictions. Assume that the number of correct predictions for each category is TP_i, and the number of wrong predictions is FP_i. Taking the accuracy rate as an example, micro average The formula for calculating the accuracy rate is:
Weighted Average (Weighted Average) is a weighted calculation based on the ratio of the number of attribute tags of each type to the total number of tags. Taking the accuracy rate as an example, the weighted average calculation formula is as follows:
Among them, n_i represents the total number of class attribute labels i, and n_total represents the total number of class attribute labels.
The principle of weighted average is the same as that of macro average, the difference is that weighted average takes more into account the proportion of the number of attribute labels of each class to the total number, while the calculation principle of micro average is similar to the calculation principle of accuracy rate.
6. Confusion Matrix
The confusion matrix (Confusion Matrix) is a method to directly express the model classification results in the form of a matrix. The rows and columns of the matrix represent the actual class attribute labels and predicted class attribute labels respectively, which are usually described in the form of drawings , through this matrix, we can intuitively understand the number of samples whose actual class attribute label is i is predicted as the number of samples whose class attribute label is j. The following figure is a common confusion matrix diagram. From this figure, we can see that the number of samples The number is directly related to the color of the matrix block, the darker the color, the larger the number of samples.
