Due to the complexity of the recommender system, there are many evaluation indicators involved. Of course, user satisfaction is the most effective, because this is the ultimate goal of the recommendation system, but the cost of limited resources is too high, and the recommendation system also relies on other objective evaluation indicators.
(1) Recommendation accuracy : This parameter can be calculated offline and is relatively objective, so it is the most important reference index for the algorithms of major research papers.
In general, the recommender system has two major tasks: "prediction" and "recommendation", so the score of the recommender system accuracy includes:
Rating prediction: learn the user's evaluation model, which is used to predict the user's rating for untouched things. In fact, it can be regarded as a regression model, which is generally measured by root mean square error or absolute error;
TopN recommendation: Give users a personalized recommendation list, which is generally evaluated by indicators such as accuracy and recall. Among them, N is also a variable parameter, and the ROC curve of the corresponding algorithm can be drawn according to different N to further evaluate the recommendation effect;
(2) Coverage rate : It reflects the ability of mining algorithms to discover long-tail commodities. The simplest definition is to combine the products recommended by all users, and then look at the proportion of the combined products that appear to the total number of products. This method is a thick line because the Matthew effect is frequent in the recommendation system. , so a good recommendation algorithm should be that all products have a similar probability of being recommended, and all suitable users can be found, so in practice, indicators such as information entropy and Gini coefficient will be considered.
(3) Diversity : The principle can be expressed as not hanging on a tree. Because there are too many factors involved in the whole recommendation system, if you only recommend similar items of one category to the user, the risk of failure is relatively high, and it is difficult to maximize the overall recommendation benefit.
(4) Novelty : The principle is that those products that the user has not contacted or manipulated, or the products with low popularity, are relatively new to the user and often have unexpected effects. Personally, I think this indicator is a bit silly~
(5) Trust degree : This indicator is relatively subjective, that is, to make users trust that the recommendation made by the recommendation system is well-founded and justified, and how the recommendation system works internally. For example, Amazon's product recommendation will give reasons for the recommendation. As a user, I will feel very intimate, otherwise the user will feel that the merchant's interests are driven and have a sense of resistance.
(6) Robustness : For example, for the associated recommendation algorithm, merchants maliciously place orders to increase the recommendation frequency of products, and navy malicious comments, etc.