Article author: Wang Peijun, senior engineer at Tencent
Editing: Li Peixin
Production platform: DataFunTalk, AI enlightener
Introduction: Angel is a distributed high-performance machine learning platform developed by Tencent, which supports scenarios such as machine learning, deep learning, graph computing, and federated learning. Angel's deep learning platform has been applied in many scenarios of Tencent. This sharing will introduce the application of Angel recommendation algorithm in game recommendation. The main contents include: game recommendation on the game platform, recommendation algorithm on the Tesla platform, the linear characteristics of the classic algorithm, the nonlinear characteristics of the DeepFM algorithm, and the application process of DeepFM.
01 Game recommendations on gaming platforms
This picture sees a game recommendation application on the Steam platform. The Steam platform is mainly a recommendation method using tags, and its tags are mainly based on the information that users choose to collect.
The characteristics of the Steam platform are that the content of the game is more and the depth is deeper.
If there are more items and more users, then some labels can be selected by the user, which can abstract the feature vector of the dependent variable that needs to be calculated by the algorithm of collaborative filtering ALS.
That is to say, Steam uses artificial selection and collective intelligence to abstract feature vectors.
This is an event on the Wegame platform. Its recommendation algorithm does not use manual extraction of labels, but uses the CF algorithm and the Deep FM algorithm based on user behavior data.
02 Recommendation algorithm on Tesla platform
This is the recommendation algorithm on the Tesla platform. If you want to try it on the Internet, you can go to the following URL to try it out:
https://cloud.tencent.com
In the trial process, generate the relevant model according to the wiki document, define the relevant parameters, and you can use it.
Some traditional algorithms can also be used on the Tesla platform, such as the CF-ALS algorithm. The important parameters of Rank, Lambda, and Alpha can be used in an iterative way to select appropriate hyperparameters. .
03 Linear Features of Classical Algorithms
If it is a content-based tag recommendation, without the abstraction of collective intelligence, it is a subjective first judgment in many cases.
The other is collaborative filtering—either item-based, or user-based, or joint collaborative filtering, which has problems with sparse matrices and long-tail recommendations.
At this point, there will be a problem: this is based on collaborative filtering with a relatively high item click rate, but most of the recommended items in the game are not all popular items. If we want to recommend some items with less clicks on the basis of the classic algorithm, that is, recommend these long-tailed items, how to do it?
At this point, you can do a clustering first, and then apply a collaborative filtering algorithm. If it is a classic algorithm, there is a big core problem: how to combine user portraits and item portraits to participate in the establishment of DeepFM?
04 Nonlinear features of DeepFM
In fact, Item Id and User ID are input in the CF algorithm, but Deep FM can also input user or item characteristics in addition to UserID and Item Id. In addition, a secondary crossover can also be performed on the features: the feature combination of the secondary label is still relatively dependent on the originally collected features, and the feature combination is only a secondary feature expression.
The various nonlinear features of DeepFM are actually similar to the weight decomposition in the recognition and classification process in the CNN algorithm. The above picture is a heat map made in the process of CNN. CNN is a classification of images. This classification basis is a part of the weight distribution previously made by Class, which was developed from a part of Deep FM's model.
The random part of the Deep FM model has many categorical features. Category features are manually selected categorical Features. In the process of automatic classification and grouping, Deep FM has an automatic embedding discrete dimension recommendation process. Generally, we can adjust the dimension combination of this feature according to the automatic adjustment of the error. Similar to the process of automatic clustering and grouping in the recommendation process, this solves the problem of traditional algorithms. If you only enter a User ID or Item ID, you cannot use user portraits and item portraits to group information. In this way, Deep FM has become a recommendation tool that can automatically adjust and cluster according to errors.
05 DeepFM application process
In the process of data operation, the commonly used tool is Vector assembler, which combines the input fields into a Vector, and then standardizes the Vector.
Another situation is to encode in the process of Deep FM. If there is a single column of category values, how to proceed and convert it into a sparse matrix? In general, hand-coding can be used, but this method is relatively inefficient. At this time, Feature Hasher can be used, which can automatically convert several columns of category values into a technical matrix, form a vector with a sparse matrix, a vector composed of continuous values, and a Hash feature, synthesize a Vector, and then use this Vector as the input of a Deep FM , it is easier to calculate the data.
Here are some experiences:
Correspondence between Validation AUC and click-through rate: that is to say, during the training process of the algorithm, a Validation AUC will be output, and this AUC will have a corresponding relationship with the click-through rate in most cases. During the model training process of the algorithm, if the AUC is relatively low, the click-through rate may be relatively high, but if the AUC is relatively high, the click-through rate is generally relatively high.
The model generated by the target data in a specific scenario can only be applied to the click-through rate evaluation in a specific scenario: you may need to use 3 collection locations. The Target data in front of these three positions is the data that the user clicks under the item. If you use 3 models, each of which is applied to 3 specific positions, you should get a better effect, and try not to use the 3 positions. The target data below is merged into one Target data. Because this is related to the feature abstraction capability of Deep FM, that is to say, Deep FM has a relatively strong feature extraction capability, and the Target data will be reversely transmitted to the feature collection process, which will be more dependent on the scene of this feature.
The parameter adjustment of Deep FM is relatively simple, because once it converges, the improvement of AUC is most closely related to the size of the training data set: because more training data is added, the improvement of AUC will be more obvious.
Data refinement needs to filter the already owned, played, and rearrange data according to business needs, such as highlighting new games, hot sales, etc.
FM needs to input user_id as training data: Deep FM is evolved from FM. During the training and use of FM, User ID is generally used as an input. The User ID of the real FM will form an input in the case of big data. big label.
The user_id of FM will form a large amount of tag data in the big data environment. For example, if there are 10 million users, input one by one, the features on the horizontal layer will be relatively large.
The user_id of FM will form the bottleneck of cold start and limit the update frequency: the traditional FM algorithm and CM algorithm must input the User ID, which will cause a cold start problem. In other words, if the user is not involved in the daily activities, there is no way to make input in the prediction stage. But Deep FM is different. If you are more confident in your own features, that means your input has many features in addition to User ID. Because Deep FM will also have relatively strong feature extraction capabilities, that is, automatic feature combination capabilities. At this time, you can not enter the User ID, only enter the characteristics of the user or the characteristics of the item, it will avoid the cold start problem, and the update frequency will be faster.
The feature input of DeepFM can replace the input of user_id: for example, when a certain amount of data is collected not long after an activity, the features are relatively abundant, and the user, User ID, Item ID, etc. have not been exposed to. data to make a prediction.
Finally, I would like to thank the students of TEG for their hard work on the tesla platform/smart titanium platform, providing an excellent spark in-memory computing platform and cutting-edge machine learning algorithms for the business!
That's all for today's sharing, thank you all.
Guest introduction:
Wang Peijun
Tencent | Senior Engineer
Wang Peijun, senior engineer of Tencent, is mainly responsible for the wegame platform's advertising system and the combination of deep learning and system exploration.
Scan the code to follow | Immediately understand Tencent's big data technology trends