1. Precision@5
Precision@5 is to evaluate the accuracy of the recommendation system in the Top-K recommendation results. The specific calculation formula is as follows:
First, according to the given recommendation system, a Top-K recommendation result list is generated for each user. Suppose there are n users, and the list of Top-K recommendation results for each user is R1, R2, ..., Rn. Then, calculate the Precision@5 value of each user. The specific calculation method is as follows:
Precision@5 = (correct number of items in the recommended list) / 5
Finally, the Precision@5 values of all users are averaged to obtain the overall Precision@5 value.
Note, only the results of the first 5 positions in the recommendation list are considered in the calculation of Precision@5, and the ranking of other positions is not considered. This is because recommendation systems tend to only focus on the first few recommendation results that are most likely to be of interest to users.
In the recommendation system, the larger the Precision@5, the better. A higher Precision@5 value indicates that the recommendation system has a higher accuracy rate in the Top-K recommendation results, that is, there are more correct items among the recommended items. A high Precision@5 value means that the recommendation system can more accurately predict the user's interests and provide users with more relevant recommendation results. This helps improve user satisfaction and user trust in the recommendation system.
However, it should be noted that Precision@5 is only an indicator to evaluate the recommendation system, and it cannot fully describe the performance of the recommendation system. When evaluating a recommendation system, other indicators such as recall, coverage, diversity, etc. need to be comprehensively considered to comprehensively evaluate the performance of the recommendation system.
2. Recall@5, NDCG@5 and MRR@5
In recommendation systems, other commonly used evaluation indicators include Recall@5, NDCG@5 and MRR@5. They represent recall, normalized loss cumulative gain, and average reciprocal rank respectively.
-
Recall@5 (recall rate): Measures the coverage rate of the recommendation system in the items actually viewed or clicked by the user, that is, how many items in the recommended list can hit the items that the user is actually interested in. The calculation formula is: Recall@5 = (number of hit items) / (number of items that the user is actually interested in).
-
NDCG@5 (Normalized Loss Cumulative Gain): Measures the ranking quality of the recommendation system in the Top-K recommendation results, that is, whether the items in the recommendation list can be sorted according to the user's true interests. The calculation formula is: NDCG@5 = (DCG@5) / (IDCG@5), where DCG@5 represents the cumulative gain of the recommended list, and IDCG@5 represents the cumulative gain under the ideal ranking.
-
MRR@5 (average reciprocal ranking): measures the ranking quality of the recommendation system in the Top-K recommendation results, that is, the reciprocal position of the items that the user is actually interested in in the recommendation list. The calculation formula is: MRR@5 = the average value of (1 / the ranking of the items that the user is actually interested in in the recommendation list).
These indicators are all for the evaluation of Top-K recommendation results, where K is generally 5. They can help evaluate the quality and performance of the recommendation system, and comprehensively evaluate the recall rate, ranking accuracy and ranking quality of the recommendation system from different perspectives.