Machine Learning - Use of Confusion Matrix, Precison, Recall, Accuracy, F1-score

Suppose there are 10 samples belonging to three categories A, B, and C. Suppose that the real and predicted categories of these 10 samples are:

Truth: AAACBCABBC

Prediction: AACBACACBC

(1) Find the confusion matrix.

(2) Find P, R, and F1 for each category.

Draft paper solution:

slightly····

Just set the formula directly, you can see the confusion matrix section in the following article.

Machine Learning - Classification Evaluation Indicators Why use ROC and AUC? Because, in the actual data set, class imbalance often occurs, that is, there are many more female samples than male samples (or vice versa), and the distribution of male and female samples in the test data may also change over time. In this case, the ROC curve can remain unchanged. https://blog.csdn.net/qq_21402983/article/details/124106002

Code solution:

import pandas as pd
from sklearn.metrics import classification_report,confusion_matrix

y_test=pd.DataFrame(['A' ,'A', 'A', 'C', 'B', 'C', 'A', 'B' ,'B', 'C'])
y_pred=pd.DataFrame(['A' ,'A', 'C', 'B', 'A', 'C', 'A', 'C', 'B', 'C'])
print(confusion_matrix(y_test,y_pred))
print(metrics.classification_report(y_test,y_pred))

 

print("查准率：",metrics.precision_score(y_test,y_pred,average=None))
print("召回率：",metrics.recall_score(y_test,y_pred,average=None))
print("F1分数：",metrics.f1_score(y_test,y_pred,average=None))

---

notes:

When I want to output each indicator separately, I get an error:

print ("Tsukuri rate:", metrics.precision_score (y_test, y_pred))
print ("Call rate:", metrics.recall_score (y_test, y_pred))
print ("F1 fraction:", metrics.f1_score ( y_test, y_pred)) )))

ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted'].

Solution
Average='micro' is added to the original code.

from sklearn.metrics import precision_score, recall_score

precision_score(y_train, y_train_pred, average='micro')

The average parameter defines the calculation method of the indicator. The average parameter is binary by default in the case of two classifications; the optional parameters are micro, macro, weighted, and samples in the case of multiple classifications.

None: Returns the score for each class. Otherwise, this will determine the type of averaging performed on the data.

binary: Only report results for the class specified by pos_label. Applies only if targets(y_{true,pred}) is binary.

micro: Calculates metrics globally by counting total true positives, false negatives, and false positives. That is, put all the classes together (specific to precision), then add the TP of all the classes, and divide by the sum of the TP and FN of all the classes. Therefore, the precision and recall under the micro method are equal to accuracy.

macro: Compute metrics for each label and find their unweighted average. This does not account for label imbalance. That is, first find the precision of each class separately and then find the arithmetic mean.

weighted: Compute metrics for each label and find their average, weighted by support (number of true instances per label). This changes the "macro" to account for label imbalance; it can cause the F-score to not be between precision and recall.

samples: Calculate metrics for each instance and find their average (only meaningful for different multi-label classification accuracy_score).

Reference: https://blog.csdn.net/datongmu_yile/article/details/81750737