Neural Graph Collaborative Filtering study notes

Paper address: https://arxiv.org/abs/1905.08108
Paper code: https://github.com/xiangwang1223/neural_graph_collaborative_filtering

Table of contents

1.Research background

2. Relevant knowledge

3. Comparison with other models

4. Dataset used in the experiment

5.Experimental results

1.Research background

The research background of this paper is that in modern recommendation systems,learning vector representations of users and items (also called embeddings) is key problem. Existing methods typically work by mapping user (or item) embeddings to pre-existing features that describe the user (or item), such as IDs and attributes. , to obtain the user (or item) embedding. However, the inherent disadvantage of this approach is that does not incorporate the potential synergistic signals (similarity) in user--item interactions ratings) are encoded into the embedding process. Therefore, the resulting embeddings may not be sufficient to capture the collaborative filtering effect. In order to solve this problem, the paper proposes a new recommendation frameworkNeural Graph Collaborative Filtering (NGCF) which willThe bipartite graph structure of user-item interaction is integrated into the embedding process by propagating the embedding on the graph. Efficiently model high-order connectivity in the user-item graph, thereby explicitly injecting collaborative signals into the embedding process. The paper provesNGCFthree public data sets 20>Compared to HOP-Rec and Collaborative Memory Network etc. Significant improvements over state-of-the-art models. Further analysis verified the importance of embedding propagation for learning better user and item representations, proving the rationality and effectiveness of NGCF.

2. Relevant knowledge

CF Two keys to modeling:

1.embedding Embedding: It converts users and items into vectorized representations

2.interaction modeling Interaction modeling: it reconstructs historical interactions based on embeddings

square decomposition ( MF) , uniform depth learning , God combination model ( NCF ), translation-based CF , etc.

None of these methods are sufficient to produce satisfactory embeddings for CF . The key reason is that the embedding function lacks explicit encoding of key collaboration signals. This encoding is hidden in user - item interactions to reveal user ( or behavioral similarity between items ) .

High connectivity:

Recommended projects foru1. The left side of the picture above is the user project interaction diagram, and the right side is based on u1 Expanded tree structure. Higher order connectivity is to u1 whose length is greater than < The node path of a i=9>1 (nodes include projects and users). This high-order connection contains rich semantics and carries collaborative signals. For example, the path u1←i2←u2 means u1 and is connected by only one path. i5 is connected by two paths, while i4 becauseu1 is more likely to interest i5 Projecti4, projectl= 3. In addition, from the overall perspective ofi4 has previously Consumedu2 because her similar useri4 Likely to adoptu1 indicates thatu1←i2←u2←i4 ;The longer pathi2 since both users have interacted with u2

The author of this article did not extend the interaction graph into a tree, because the tree is more complicated to implement. He designed a hierarchical propagation layer to aggregate interactive items(< Embedding of /span>( to refine users) or user). By stacking multiple embedding propagation layers, embedding can be forced to capture cooperative signals in higher-order connections.

The author summarizes the following three main contributions:

1, emphasized the importance of model-based CF The critical importance of explicitly leveraging collaborative signals within the embedded functionality of the approach.

2, proposed a new recommendation framework based on graph neural networkNGCF a>, which explicitly encodes collaboration signals in the form of higher-order connections by performing embedding propagation

3, on 300 million data sets An empirical study was conducted. A large number of results demonstrate the advanced performance of NGCF and its effectiveness in improving the quality of neural embedding propagation.

NGCFModel architecture diagram:

Useu1(Left)Japanese item a>i4(Right)The display can be passed in many places.)line display information flow(

Frame composition:

(1)Embedding layer:Provides and initializes user embeddings and project embeddings;

(2)Multiple embedded propagation layers:Refined by injecting higher-order connection relationships Embed;

(3)Prediction layer:This layer aggregates refined embeddings from different propagation layers, And output the similarity scores of user- items.

More forms of embedding the propagation layer (only one layer was just discussed):

1, High-order propagation: Stack more embedded propagation layers to explore higher-order connectivity sexual information. (formulated recursively)

2,square format 传行规则：

graph Laplacian matrix

Optimizing PairwiseBPRLoss:

AmongO={(u, I, j)|(u, i)∈ R+ , (u, j)∈R- }Display configuration, R+Display observation and interaction, R-Interactions that have not been observed;

σ()issigmoidfunctionθ= { E,{W(l) 1,W(l) 2 }L l=1}Display ownable model number,

λrestraintL2Prevention of correcting the strength.

Use mini-batchesAdam to optimize the prediction model and update model parameters.

Specifically, for a batch of randomly sampled triples(u, Ie(L)], and then use the gradient of the loss function to update the model parameters. ,,[e(0)Establish their representations after step propagationL, we are at j)∈ O,

Message and node lost:

Although deep learning models have strong representation capabilities, they often suffer from overfitting problems. In order to solve this problem, dropout is an effective solution. to prevent the neural network from overfitting. NGCFadopts two discarding techniques:message discarding and node discarding. Message Dropping: Specifically, we drop equation(6)p1 =10> message spread in. Therefore, in the lpropagation layer, only part of the message contributes to the refined representation. Node Drop: Randomly blocks a specific node and discards all its output messages. For the l propagation layer, we randomly discard the (M+N) only applies to training and cannot be used during testing. Role: Message loss makes the representation more robust when a single connection exists or does not exist between users and projects, node loss focuses on reducing the impact of a specific user or project. dropout is discarded Rate. Note: p2 nodes, wherep2

3. Comparison with other models

NGCF has the following advantages compared with other models:

1. Compared with traditional collaborative filtering methods (such as MF , NeuMF can Better capture the complex and non-linear relationships between users and items, thereby providing more accurate recommendation results. NGCF), CMN and

2. Compared to models based on graph convolutional networks (such as GC-MC and PinSage ), NGCF are better able to capture users by explicitly encoding collaborative signals during the embedding process - Higher-order connectivity in the item graph, thereby providing a more comprehensive representation to infer user preferences.

3. Compared with attention mechanism-based models (such as HOP-Rec ), NGCF By stacking multiple embedding propagation layers, it is able to explore explicit high-order connectivity and provide multi-granularity representation to infer user preferences.

4. Compared with the traditional linear inner product method (such as MF ), NGCF is able to more accurately predict interactions between users and items by explicitly encoding collaborative signals during the embedding process.

4. Dataset used in the experiment

Experiments were conducted on three benchmark datasets: Gowalla, yelp 2018 * 2, these datasets are publicly accessible and vary in domain, size, and sparsity. Are not the same. Amazon-book and

Gowalla:This is the check-in data set obtained from Gowalla, where users share their Location. To ensure the quality of the dataset, we use the 10 core setting, which retains users and items with at least ten interactions.

Yelp2018:This data set adoptsYelpChallengeData from the 2018 version. Among them, local businesses like restaurants and bars are considered projects. To ensure data quality, we used the same10core settings.

Amazon-book: Amazon-review is a dataset widely used for product recommendation[9]. We select Amazon books from our collection. Again, we use a ten-core setup to ensure at least ten interactions per user and project.

For each data set, we randomly select each user80% of the historical interactions to form the training set, and the remaining ones are used as the test set. From the training set, we randomly select 10% of interactions as validation set to adjust the hyperparameters.

5.Experimental results

UseNGCDSeparate sumMF,HOP-Rec The author's thoughts are clear as the comparison progresses. GC-MCsumPinSage, , CMN, NeuMF

NGCF consistently produces the best performance on all datasets. By stacking multiple embedding propagation layers, NGCF is able to explore higher-order connectivity in an explicit manner, while CMNBetter representation can be obtained. CF showing explicit encoding in embedded functionsHOP-Reconly uses the output of the last layer. This suggests that different propagation layers encode different information in the representation. Improvements toPinSage considers multi-granularity representation for inference user preference, whileNGCF, PinSage only utilize first-order neighbors to guide representation learning. This validates the importance of capturing cooperative signals in embedded functions. Furthermore, compared to GC-MC and

Performance comparison of user group sparsity distribution on different data sets. where the background histogram represents the number of users involved in each group and the lines represent the performance of w.r.t. ndcg@20 .

NGCF and HOP-Rec consistently outperform all other benchmarks on all user groups . It is shown that exploiting higher-order connectivity greatly facilitates representation learning for non-active users since collaborative signals can be efficiently captured. Therefore, it is promising to solve the sparsity problem in recommender systems.

NGCFThe results of different layers show better results at the third and fourth layers

node dropout provides better performance. One reason could be that discarding all outgoing messages from a specific user and project makes the representation not only immune to specific edges, but also node-independent. Therefore, node loss is more efficient than message loss. It meansnode dropout can be an effective strategy to solve the overfitting of graph neural network.

Each iteration,NGCF performs better than MF Faster convergence when interacting pairs are optimized in mini-batches involving indirectly connected users and projects. Such observations demonstrate the better model capacity of NGCF and the effectiveness of performing embedding propagation in the embedding space.

Experiences derived fromMF andNGCF-3 t-SNEVisualization of transformation representation. 6 users were selected, each star represents a data from Gowalla Set users, while dots with the same color represent related items. (Observe the degree of aggregation of blue and yellow)

Replenish:

t-SNE (t-Distributed Stochastic Neighbor Embedding) is a method for dimensionality reduction and Machine learning algorithms for visualizing high-dimensional data. It is a nonlinear dimensionality reduction technique commonly used to map high-dimensional data to a low-dimensional space for visualization or feature extraction. The main goal of t-SNE is to preserve the similarity relationship between data points, especially the relative distance information that is difficult to capture in high-dimensional space.