Classification results confusion matrix (confusion matrix):
| Real \ forecast | Positive example | Counterexample |
| Positive example | TP | FN |
| Counterexample | FP | TN |
1. accuracy --accuracy
Definitions: For a given set of test data, than the number of samples correctly classified and the classification number of the total sample.
Calculation method:
2. The exact rate --precision (P)
is defined: Example is determined to be positive (negative examples) sample, the ratio of true positive samples (negative samples) of.
Calculation method:
3. Recall --recall (R)
is defined: the ratio of positive examples (Example trans) samples, representing all positive examples (Example trans) samples are correctly classified.
Calculation method:
4.F1_score
defined: based on a harmonic mean of precision and recall.
Calculation method:
5.macro metric
definitions: For n binary confusion matrices, each confusion matrix calculation precision and recall are denoted (P1, R1), (P2 , R2) ... (Pn, Rn), recalculation the average rate to obtain accurate macro (macro-P), recall the macro (macro-R), and then to give the macro F1 (macro-F1).
6.micro metric
definitions: For n binary confusion matrix, the first averaged TP, FN, FP, TN, and then calculate the mean accuracy of micro (micro-P), recall micro (micro-P), then give a slightly F1 (micro-F1).
# - * - Coding: UTF-. 8 - * - Import numpy # to true = [1 real group, group 2 ... real real groups N], predict = [1 prediction set, the prediction set group 2 ... prediction N] DEF Evaluation (to true, Predict): NUM = len (to true) # determines several groups (TP, the FP, FN, the TN) = ([0] * NUM for I in Range (. 4)) # initial value for m in the Range (0, len (to true)): IF (! len (to true [m]) = len (predict [m])): # number of samples, etc. are not clearly erroneous Print " real results with predicted results of samples number of inconsistencies. " the else : for I in Range (0, len (to true [m])): #Were counted for each set of data IF (Predict [m] [I] ==. 1) and ((to true [m] [I] ==. 1)): TP [m] = 1.0 + elif (Predict [m] [ I] ==. 1) and ((to true [m] [I] == 0)): the FP [m] = 1.0 + elif (Predict [m] [I] == 0) and ((to true [m] [ I] ==. 1)): FN [m] = 1.0 + elif (Predict [m] [I] == 0) and ((to true [m] [I] == 0)): the TN [m] + = 1.0 # Macro metric, calculates an evaluation index of each first group, and then averaging (accuracy_macro, \ precision1_macro, precision0_macro, \ recall1_macro, recall0_macro, \ F1_score1_macro, F1_score0_macro) = \ ([0] * num for i in range(7)) for m in range(0,num): accuracy_macro[m] = (TP[m] + TN[m]) / (TP[m] + FP[m] + FN[m] +TN[m]) if (TP[m] + FP[m] == 0) : precision1_macro[m] = 0#预防一些分母为0的情况 else :precision1_macro[m] = TP[m] / (TP[m] + FP[m]) if (TN[m] + FN[m] == 0) : precision0_macro[m] = 0 else :precision0_macro[m] = TN[m] / (TN[m] + FN[m]) if (TP[m] + FN[m] == 0) : recall1_macro[m] = 0 else :recall1_macro[m] = TP[m] / (TP[m] + FN[m]) if (TN[m] + FP[m] == 0) : recall0_macro[m] = 0 recall0_macro[m] = TN[m] / (TN[m] + FP[m]) macro_accuracy = numpy.mean(accuracy_macro) macro_precision1 = numpy.mean(precision1_macro) macro_precision0 = numpy.mean(precision0_macro) macro_recall1 = numpy.mean(recall1_macro) macro_recall0 = numpy.mean(recall0_macro) #F1_score还是按这个公式来算，用macro-P和macro-R if (macro_precision1 + macro_recall1 == 0): macro_F1_score1 = 0 else: macro_F1_score1 = 2 * macro_precision1 * macro_recall1 / (macro_precision1 + macro_recall1) if (macro_precision0 + macro_recall0 == 0): macro_F1_score0 = 0 else: macro_F1_score0 = 2 * macro_precision0 * macro_recall0 / (macro_precision0 + macro_recall0) #micro度量，是用TP、TN、FP、FN的均值来计算评价指标 TPM = numpy.mean(TP) TNM = numpy.mean(TN) FPM = numpy.mean(FP) FNM = numpy.mean(FN) micro_accuracy = (TPM + TNM) / (TPM + FPM + FNM + TNM) if(TPM + FPM ==0): micro_precision1 = 0#Some prevention denominator is zero the else : the TPM micro_precision1 = / (the TPM + FPM) IF (the TNM + FNM. == 0): micro_precision0 = 0 the else : the TNM micro_precision0 = / (the TNM + FNM.) IF (the TPM + == 0 FNM. ): micro_recall1 = 0 the else : the TPM micro_recall1 = / (the TPM + FNM.) IF (FPM the TNM + == 0): micro_recall0 = 0 the else : the TNM micro_recall0 = / (the TNM + FPM) # F1_score still calculated according to the formula, with and micro-P-R & lt Micro IF (+ micro_recall1 micro_precision1 == 0): micro_F1_score1 = 0 the else :micro_F1_score1 = 2 * micro_precision1 * micro_recall1 / (micro_precision1 + micro_recall1) if (micro_precision0 + micro_recall0 == 0): micro_F1_score0 = 0 else :micro_F1_score0 = 2 * micro_precision0 * micro_recall0 / (micro_precision0 + micro_recall0) print "*****************************macro*****************************" print "accuracy",":%.3f" % macro_accuracy print "%20s"%'precision',"%12s"%'recall',"%12s"%'F1_score' print "%5s" % "0", "%14.3f" % macro_precision0, "%12.3f" % macro_recall0, "%12.3f" %macro_F1_score0 print "%5s" % "1", "%14.3f" % macro_precision1, "%12.3f" % macro_recall1, "%12.3f" %macro_F1_score1 print "%5s" % "avg","%14.3f" % ((macro_precision0+macro_precision1)/2), \ "%12.3f" % ((macro_recall0+macro_recall1)/2), "%12.3f" %((macro_F1_score1+macro_F1_score0)/2) print "*****************************micro*****************************" print "accuracy",":%.3f" % micro_accuracy print "%20s"%'precision',"%12s"%'recall',"%12s"%'F1_score' print "%5s" % "0", "%14.3f" % micro_precision0, "%12.3f" % micro_recall0, "%12.3f" %micro_F1_score0 print "%5s" % "1", "%14.3f" % micro_precision1, "%12.3f" % micro_recall1, "%12.3f" %micro_F1_score1 print "%5s" % "avg", "%14.3f" % ((micro_precision0 + micro_precision1) / 2), \ "%12.3f" % ((micro_recall0 + micro_recall1) / 2), "%12.3f" % ((micro_F1_score0 + micro_F1_score1) / 2) if the __name__ == " __main__ " : # Simple Example - When not using the cross-validation method, the True and apparent Predict have only one set, the value of Macro and Micro outputs the same true = [[0, 1, 0, 1, 0] , [0,. 1,. 1 , 0]] Predict = [[0,. 1,. 1,. 1, 0], [0,. 1, 0,. 1 ]] Evaluation (to true, Predict)
*****************************macro***************************** accuracy :0.650 precision recall F1_score 0 0.750 0.583 0.656 1 0.583 0.750 0.656 avg 0.667 0.667 0.656 *****************************micro***************************** accuracy :0.667 precision recall F1_score 0 0.750 0.600 0.667 1 0.600 0.750 0.667 avg 0.675 0.675 0.667