Guest speaker: Sun Ruihong Tencent Big Data
Editing: Zhao Wenjiao
Production platform: DataFunTalk, AI enlightener
Introduction: With the development of data diversity, graph computing has become an important research direction in the industry. Among them, graph neural networks are widely used in graph representation learning. Compared with traditional graph learning, it can learn the topology of graph networks. , and can also aggregate neighbor features, so that the information in the graph network can be effectively learned, which plays a key role in the subsequent recommendation work.
01 Introduction to Graph Neural Networks
- Internet of Everything
Taking the construction of social networks as an example, it was mainly based on "following relationships" before, but now it will take into account data such as "likes, shares, and comments" in multiple directions. The practical application of graph algorithms has been widely used in various scenarios that are more widely used in Tencent.
- Application scenarios
Recommendations based on key figures. Through some graph structures combined with some algorithms, such as the typical pagerank algorithm, we can find key figures, and improve the recommendation effect by adopting specific strategies (such as targeted promotion) for key figures.
Circle-based recommendations. Recommend products and advertisements based on circles composed of geography, person-task relationship, and hobbies.
- Recommended Graph Mining Algorithms
Graph mining algorithms, including traditional graph mining algorithms, such as pagerank, closeness, etc.; community discovery algorithms for finding key nodes and key groups, including graph representation learning algorithms deepwalk, node2vec, etc. Today we mainly introduce graph neural networks usage of.
The graph neural network considers more, considering the characteristics of nodes, the characteristics of edges, and learning by aggregating the characteristics of neighbors and combining their own characteristics. The effect of the graph neural network is also obvious, and the effect in different scenarios is as follows.
02 Angel Graph Neural Network
- What is a Graph Neural Network
A graph neural network is the application of a neural network framework on a graph. The similarities and differences between graph neural network and neural network, for example: the data processed by traditional neural network framework is based on standard Euclidean space, such as image data, text data, etc.; for graph neural network, it needs to process graph network topology data, which is a non- The Euclidean space data is as follows. The picture on the left is a common cat picture in image classification. It is a standard Euclidean space. The graph network on the right is a non-Euclidean space. Very difficult to scale. Due to this difference in the data structure, the graph neural network cannot follow the processing method of the neural network, but the graph neural network constructs features by aggregating the features of neighbor nodes.
- Angel Graph Neural Network Framework
There is no special detailed explanation here, and the official documentation can be used for detailed study. It is emphasized here that there are first-order neighbors and second-order neighbor features when aggregating neighbor features. The first-order neighbor features are placed on the spark executor, and the sampling speed is fast. Then the adjacency list will be put on the ps to facilitate the second-order sampling server.
Angel's existing graph neural network algorithms have many graphsages, which are divided into supervised, unsupervised, isomorphic, and heterogeneous.
Let's take graphsage as an example to introduce our concerns.
GraphSage principle:
GraphSage has two key points, one is sampling and the other is aggregation, as indicated by the arrows in the figure below.
sampling. Select a point, first randomly sample its first-order neighbors, and then sample second-order neighbors according to the first-order neighbors obtained by sampling, and so on. At present, our experiments have found that second-order neighbor sampling can get good results. After getting the neighbors, let's start thinking about how to aggregate neighbor features.
polymerization. First, the second-order neighbor features are aggregated, then the aggregated second-order neighbor features and the first-order neighbor features are fused, and finally the fused features are fused with the features of the initial node. In this way, an embedding of fusion features is obtained.
03 Practice in Recommended Scenarios
- Overview of GNN Recommendation Scenarios
The first is to build a network, for example, through social networks, mobile payments, communication networks, e-commerce shopping and other scenarios to build a network structure, and then put it into gnn to get the embedding, and then finally it can be applied to more downstream recommendations, such as as Model features, recommended recall, recommended cold start, user portraits, etc. Below we introduce the specific scenarios based on the scenario:
Scenario 1: Tencent Watch Video Recommendations
Build a network from a user's viewing history. Specifically, each user has his own viewing record, and from the perspective of video, we can get the network structure diagram as shown below. This process has information loss, that is, the user's viewing history is incomplete. We try to find a way solve this problem.
The first step is network construction. As mentioned above, the additional features that need to be added include user features, video features, and viewing record features. We do not want to lose sequence features, so we will use some sequence algorithms, such as transformerTransformer to process historical data of the sequence , you can get the user's embedding, and then put the video's embedding into the graphGraph-sageSAGE for learning, so that the last thing learned includes both the user's historical viewing records and the information of the network structure.
Finally, the semi-supervised GraphSAGE graphsage is used, which acts on tens of millions of points and billions of edges. In the offline table effect, compared with the direct transformer, the top50 hit rate is increased by about 3%. In the online indicator, the short video duration is increased by 0.98%, and the total duration is increased by 0.34%.
Scenario 2: WeChat content recommendation
This scenario is to recommend official accounts for WeChat users. First, build a network. If you directly use the attention relationship to build a network, there is a problem. As shown in the figure below, we found that a super node such as "People's Daily" appeared, and too many users paid attention to others. The solution is to convert the graph from the relationship between the user and the official account to the relationship between the official account and the official account. picture.
This kind of change will also bring about other problems, for example: the super-large size will be connected with many other points, which will also cause problems. To solve this problem, you need to deliberately do some side screening. Considering that attention is only an action, we can also add actions such as watching and clicking as edge trigger actions.
Feature engineering includes: features of official accounts, user features, and denoising of these features.
Algorithm choice: We could choose GraphSAGEgraphsage or DGI, but in the end we chose DGI.
Regarding the selection of first-order and second-order, it is finally determined that the first-order effect is better, and the second-order is easy to form embedding convergence.
The process is as follows:
After using dgi to extract features, use itemCF to select some seeds, delineate a recall from the full public account, and then perform a series of behaviors such as sorting.
Finally, use millions of points and billions of edges. The exposure rate increased by 2.38%, the click rate increased by 1.61%, and the attention rate increased by +0.5%.
04 Experience Summary
Network construction and feature engineering have a key impact on the task objective, and mainly need to pay attention to noise reduction.
The algorithm is not universal and needs to be adjusted according to the scene. The effect of multi-model fusion is generally better. We have used:
gnn+transformer,Gnn+xgboost,gnn+kmeans,gnn+itemcf等。
That's all for today's sharing, thank you all.
Guest introduction:
Sun Ruihong
Tencent Big Data | Senior Algorithm Engineer
Sun Ruihong, Senior Algorithm Engineer of Tencent Big Data, currently focuses on the research and development of Angel graph algorithm and the implementation of Tencent's internal scenes.
Scan the code to follow | Immediately understand Tencent's big data technology trends