Recommended evaluation indicators commonly used in the system: HR, NDCG, MRR

Recommendation system commonly used evaluation indicators

Application background introduction

We use a table to display the list recommended by the recommendation system to the user and the user's actual visit items, as follows:

user	actual value	Recommended list
A	12	3,10,15,12,17
B	3	20,15,18,14,30
C	5	2,5,7,8,15
D	14	56,14,25,12,19
E	20	21,24,36,54,45

Knowledge sources of evaluation indicators

The source of the evaluation index is based on the method used in a paper in TKDE-2019:
Insert picture description here

HR (Hits Ratio)

Significance: I care about what the user wants. Have I recommended it? Emphasize the "accuracy" of the prediction
$HR=\frac{1}{N}\sum_{i=1 }^{N}{hits(i)}$
parameter description:
N: total number of users
hits(i): whether the value accessed by the i-th user is in the recommended list, if yes, it is 1, otherwise it is 0

Normalized Discounted Cumulative Gain (NDCG)

Meaning: care about whether the items found are placed in a more conspicuous position for users, that is, to emphasize the "sequence"
$NDCG=\frac{1}{N }\sum_{i=1}^{N}\frac{1}{log_{2}{(p_{i}+1)}}$
Parameter description:
N: total number of users
$p_{i}: the real access value of the i-th user In the position of the recommendation list, if the value does not exist in the recommendation list, then p_{i}\to\infty$

Mean Reciprocal Rank (MRR)

Meaning: care about whether the items found are placed in a more conspicuous position for users, that is, to emphasize "sequence"
$MRR=\frac{1}{N}\sum_{i= 1}^{N}\frac{1}{ {p_{i}}}$
Parameter description:
N: total number of users
$p_{i}: the real access value of the i-th user In the position of the recommendation list, if the value does not exist in the recommendation list, then p_{i}\to\infty$

Instance

user	actual value	Recommended list
A	12	3,10,15,12,17
B	3	20,15,18,14,30
C	5	2,5,7,8,15
D	14	56,14,25,12,19
E	20	21,24,36,54,45

code show as below:

import math

# 推荐列表
R = [[3, 10, 15, 12, 17], [20, 15, 18, 14, 30], [2, 5, 7, 8, 15], [56, 14, 25, 12, 19], [21,24,36,54,45]]
# 用户访问列表
T=[[12],[3],[5],[14],[20]]


def indicators_5(rankedList, testList):
    Hits_i = 0
    Len_R = 0
    Len_T = len(testList)
    MRR_i = 0
    HR_i = 0
    NDCG_i = 0
    for i in range(len(rankedList)):
        for j in range(len(rankedList[i])):
            if testList[i][0]==rankedList[i][j]:
                Hits_i+=1
                HR_i+=1
                # 注意j的取值从0开始
                MRR_i+=1/(j+1)   
                NDCG_i+=1/(math.log2(1+j+1))
                break
    HR_i/=Len_T
    MRR_i/=Len_T
    NDCG_i/=Len_T
    print(Hits_i)
    print(f'HR@5={HR_i}')
    print(f'MRR@5={MRR_i}')
    print(f'NDCG@5={NDCG_i}')



if __name__ == '__main__':
    indicators_5(R, T)

If it works for you, please like it, thank you!

reference

1.https://blog.csdn.net/qq_34862636/article/details/105010511
2.https://blog.csdn.net/Allenalex/article/details/78161915