Several indicators for machine learning and deep learning classification problems: accuracy, precision, recall, F1-score, Macro-F1, Micro-F1

table of Contents

4. Further explore the recall rate and accuracy rate

Before talking about these indicators, there is a very important thing called the confusion matrix:

We call the above-mentioned table a confusion matrix, so why do we generally use a confusion matrix? Calculate the recall rate, calculate the accuracy rate, and calculate the accuracy rate. Next, we will introduce these three indicators separately.

TP: Actually a positive sample is predicted as a positive sample; TN: A negative sample is actually predicted as a negative sample ; FN: Actually a negative sample is actually predicted as a negative sample ; FP: A negative sample is actually predicted as a positive sample

1. Accuracy-accuracy rate

The prediction is the proportion of the correct number of samples to the total number of samples.

$Acc = \frac{TP+TN}{TP+FN+FP+TN}$

2. Precision——Accuracy rate

The accuracy rate is only related to the predicted positive samples, which is defined as how many of the samples predicted to be positive are truly positive samples .

$precision=\frac{TP}{TP+FP}$

3. Recall-recall rate

The recall rate is only related to the prediction of the positive sample, which is defined as how many positive examples in the sample are predicted correctly .

$Recall=\frac{TP}{TP+FN}$

After having a clear definition of these indicators, I can't help but ask why these indicators are used? Measure the classification or detection effect of the model. So how to measure it, it is necessary to further explore the recall rate and accuracy rate.

4. Further explore the recall rate and accuracy rate

For extreme classification tasks, if only one result is found and the result is correct, then the accuracy rate is 100%, and the recall rate is very low. Conversely, if the recall rate is very high, then the accuracy rate will be relatively low. From this we can see that these two are relatively contradictory and there is no way to unify them. Of course, we hope that our model has a better accuracy rate and also a better recall rate. This is to find a balance indicator, at this time we can lead to another balance indicator, called F1-Score.

5、F1-Score

F1-score is an indicator to measure the accuracy of a two-class model . It takes into account both the accuracy and recall of the classification model. F1-score can be regarded as a harmonic average of model quasi-deletion rate and recall rate. The maximum value is 1 and the minimum value is 0.

$F1-score=2*\frac{Precision*Recall}{Precision+Recall}$

Next, introduce Micro-F1 and Macro-F1

6、Micro-F1和Macro-F1

If for a multi-label classification task, then we need to calculate the recall rate and accuracy rate of each class. For calculating F1-score, we obviously combine all categories of indicators for consideration.

Micro-F1: Calculate the total recall rate and accuracy rate without distinguishing categories. Taking into account the number of each category, it is suitable for the case of unbalanced data distribution.

Macro-F1: Calculate the recall rate and accuracy rate of each category, and then average the accuracy and accuracy rates of each category. Then finally F1-score is calculated. Without considering the amount of data, treating each category equally will be relatively more affected by the high precision and high recall categories.

7. Example calculation description

I. Calculate the confusion matrix for each category

①A

②B

③C

With three confusion matrices, the accuracy and recall rates of the three categories can be calculated, and then the Macro-F1 score can be calculated.

2. Calculate the overall confusion matrix

Through the overall confusion matrix, the recall rate and accuracy rate can also be calculated, and then the opportunity to get Micro-F1 score.

The source of this example Macro-F1 Score and Micro-F1 Score

The use of these indicators in target detection and cross-modal retrieval will be introduced later...