Common multi-model fusion algorithms
The multi-model fusion algorithm can significantly improve the effect of the single-model algorithm. But how to effectively integrate and give full play to the strengths of each algorithm? Here are some common fusion methods:
1. Linear weighted fusion method
Linear weighting is the most simple and easy-to-use fusion algorithm. It is very convenient for engineering implementation. It only needs to summarize the results of a single model, and then assign different weights according to different algorithms, and weight the results of multiple recommendation algorithms to obtain the results:
is the score of recommending a product (item) to a user (user), is the weight of the algorithm K, and is the user's (user) recommendation score for the product item obtained by the algorithm k. This fusion method is simple to implement, but the effect is poor. Because the parameters of linear weighting are fixed, in practice, the selection of parameters usually depends on the summary of the global results. Once set, it cannot be flexibly changed automatically according to different recommended scenarios. For example, if algorithm A is used for a certain scene, and algorithm B is used for another scene, the linear fusion method cannot achieve good results in this case. In order to solve this problem, Optimistic Data has been improved. By introducing the mechanism of dynamic parameters, a weighted model is generated by training the user's evaluation of the recommended results and whether it is consistent with the system's prediction, and the dynamic adjustment of the weight greatly improves the effect.
2. Cross-Fusion Method
Cross-fusion is often referred to as the Blending method. The idea is to intersperse the results of different recommendation models in the recommendation results to ensure the diversity of results.
In this way, the results of different algorithms are combined and recommended to the user.
The idea of the cross-fusion method is "each flower enters each eye". The results of different algorithms have different points of view, which can meet the needs of different users, and are directly interspersed for display. This fusion method is suitable for recommendation scenarios that can display many results at the same time, and is often used for algorithms with large differences, such as the results obtained based on long-term interests and short-term interests of users respectively.
3. Waterfall fusion method
The waterfall (Waterfall Model) fusion method adopts the method of connecting multiple models in series. Each recommendation algorithm is treated as a filter by chaining filters of different granularities together:
In the waterfall hybrid technology, the results filtered by the previous recommendation method will be used as the input of the candidate set of the latter recommendation method, and the candidate results will be gradually selected in this process, and finally a small quantity and high quality will be obtained. result set. This design is usually used in recommendation scenarios where there are a large number of candidate sets.
In designing a waterfall hybrid system, algorithms with fast computing speed and low discrimination are usually placed at the forefront, and gradually transition to heavyweight algorithms, so that precious computing resources are concentrated on a small number of higher candidate results. In the face of a large number of candidate recommendation objects (Items), but few recommended results that can be exposed, it is often very suitable for scenarios that require high precision and limited computing time.
4. Feature fusion method
Different raw data quality has a great influence on the result of recommendation calculation. Taking the user interest model as an example, we can not only mine the user's "explicit" interest from the user's actual purchase behavior, but also use the user's click behavior to mine the user's "implicit" interest; In demographic analysis, user preferences can also be inferred; if there is a user's social network, then the influence of surrounding users on the user's interest can also be understood.
So by using different data sources, extract different features, input them into the recommendation model for training, and then combine the results. This kind of thinking can solve the problem of missing data that is often encountered in reality, because not all users have complete data of various types, for example, some users lack transaction information, and some do not have social relationship data. The method of feature fusion can ensure that the model is not picky eaters and expand the scope of application.
5. Predictive fusion method
A recommendation algorithm can also be thought of as a "prediction algorithm", i.e. we predict for each user which item he is most likely to like next. The idea of the prediction fusion method is that we can make another prediction for each prediction algorithm, that is, the prediction results of different algorithms. We can train the prediction algorithm of the second layer to make predictions again and generate the final prediction results.
As shown in the figure below, we use the prediction results of each recommendation algorithm as a feature, and use the user's feedback data on the product as a training sample to form a training set of the second-level prediction model. The specific process is as follows
The two-layer prediction model in the figure can use commonly used classification algorithms, such as SVM, random forest, large entropy, etc., but in philosophical practice, the better fusion effect is the GBDT (Gradient Boosting Decision Tree) method.
6. Classifier Boosting idea
The recommendation problem can sometimes be transformed into a pattern classification problem. We divide whether the candidate set is worth recommending into several different sets, and then solve it by designing a classifier.
In this way, we can use the Boosting idea in the classification algorithm, that is, a method of combining several weak classifiers into a strong classifier. The core idea of Boosting is to assign a larger weight to the wrongly predicted samples after each round of training, and add them to the subsequent training set, that is, let the learning algorithm perform reinforcement learning on the more difficult cases in the subsequent training set, so as to obtain a weighted sample. The prediction function sequence h, the prediction function with good prediction effect has a larger weight, and vice versa.
The final prediction function H uses a weighted voting method for classification problems and a weighted average method for regression problems to discriminate new examples. The flow of the algorithm is as follows: (referenced from the treeBoost paper)
Fusion through models is often effective, but the implementation cost and computational overhead are also relatively large.