Commonly used evaluation indicators for deep learning (Accuracy, Recall, Precision, HR, F1 score, MAP, MRR, NDCG) - recommendation system

confusion matrix

confusion matrix
	P(Positives)	N(Negatives)
T(Ture)	TP: Positive sample, the predicted result is positive	TN: Negative sample, the predicted result is positive
F(false)	FP: Positive sample, the prediction result is negative	FN: Negative sample, the prediction result is negative

Summarize

Accuracy	$\frac{TP+TN}{TP+FP+TN+FN}$
Recall	$\frac{TP}{TP+FN}$
Precision	$\frac{TP}{TP+FP}$
Hits Ratio	$HR=\frac{1}{N}\sum\limits_{i=1}^{N}{hits(i)}$
F1 score	$F1score=2\frac{precision\cdot recall }{precision+ recall} \\ =\frac{2TP}{2TP+FP+FN}$
Mean Average Precision	$AP=\frac{1}{R}\sum{Precision(rank)} \\ MAP=\frac{1}{C}\sum{AP}$
Mean Reciprocal Rank	$MRR=\frac{1}{N}\sum\limits_{i=1}^{N}{\frac{1}{{{p}_{i}}}}$
Normalized Discounted Cumulative Gain	$NDCG=\frac{1}{N}\sum\limits_{i=1}^{N}{\frac{1}{lo{{g}_{2}}({{p}_{i}}+1)}}$

1. Accuracy _

Meaning: The proportion of correctly predicted samples among all samples.

official: $\frac{TP+TN}{TP+FP+TN+FN}$

Note: In the case of unbalanced samples, it is not a good indicator to measure the results.

2. Recall & Recall

Meaning: The proportion of correctly predicted positive samples in all positive samples, that is, how many user-item interaction records are included in the final prediction list.

official: $\frac{TP}{TP+FN}$

Note: Both precision and recall measure only one aspect of retrieval performance.

3. Precision & Precision

Meaning: Among all the predicted positive results, the proportion of positive samples that are predicted to be correct.

official: $\frac{TP}{TP+FP}$

Note: The recall rate and the precision rate are a pair of contradictory indicators. When the recall rate is high, the precision rate is generally very low; when the precision rate is high, the recall rate is generally very low.

4. Hit rate (Hits Ratio)

Meaning: The ratio of correctly predicted samples to all samples in the predicted result list, that is, whether the item that the user wants has been recommended, emphasizing the "accuracy" of the prediction.

official: $HR=\frac{1}{N}\sum\limits_{i=1}^{N}{hits(i)}$

N, represents the total number of users .

hits(i), indicates whether the value accessed by the i - th user is in the recommendation list, if it is, it is 1 , otherwise it is 0 .

5. F1 score (F1 score)

Meaning: The harmonic mean of precision and recall.

official: $F1score=2\frac{precision\cdot recall }{precision+ recall}=\frac{2TP}{2TP+FP+FN}$

6. Mean Average Precision

Average Precision (AP): The average precision rate, while the recall rate gradually increases from 0 to 1, it is also necessary to ensure that the accuracy rate is relatively high, and the AP value is as large as possible.

Meaning: Use multiple to measure performance, the average of multiple categories of AP.

official: $AP=\frac{1}{R}\sum{Precision(rank)} \\ MAP=\frac{1}{C}\sum{AP}$

7. Mean Reciprocal Rank

Meaning: The reverse order of the average results, indicating whether the item to be recommended is placed in a more prominent position for the user, emphasizing "sequence".

official: $MRR=\frac{1}{N}\sum\limits_{i=1}^{N}{\frac{1}{{{p}_{i}}}}$

N, represents the total number of users.

pi, indicating that the real access value of the i-th user is in the recommendation list, if the value does not exist in the recommendation list, then pi->∞.

8. Normalized Discounted Cumulative Gain

Meaning: Indicates the accumulative benefits of the first p positions that are normalized and added to the position information measurement calculation.

official: $NDCG=\frac{1}{N}\sum\limits_{i=1}^{N}{\frac{1}{lo{{g}_{2}}({{p}_{i}}+1)}}$

N, represents the total number of users.

pi, indicating that the real access value of the i-th user is in the recommendation list, if the value does not exist in the recommendation list, then pi->∞.