Using deep learning methods to extract hidden user item features from user ID, comment text and other data, and predicting users' ratings for new items based on these features to give recommendations is the mainstream approach of traditional recommendation algorithms

at this stage

After the graph neural network was proposed, researchers found that the graph neural network can be used to extract the connection characteristics between users and items that cannot be extracted in the traditional recommendation algorithm, and use the graph neural network to build a recommendation algorithm

limitations

At present, most recommendation algorithms based on graph neural networks use the ID information of user items to extract user item features, ignoring the user item features implied by other data such as comment text in the data, or using auxiliary information to make network training more complicated. .

innovation

This paper proposes a graph neural network-based recommendation algorithm that combines traditional deep learning to fuse the features of different user items.

method

Utilize the LightGCN base network while adding an attention mechanism to extract high-order features between user items as part of the user item feature representation.
Another part of user-item features is extracted from review text using a traditional deep learning network.
Finally, the two feature representations are fused to obtain the end user item feature representation, and then recommendations are given.

result

The experimental results show that this method can better integrate the high-order features and general characteristics of user items, so that the features of user items can be better expressed, and the recommendation effect for users can be effectively improved.

This paper proposes a recommendation algorithm based on graph neural network that integrates traditional deep learning, and extracts user features through traditional deep learning methods and graph neural network, so as to better express the user's interest preferences; using the attention mechanism, for Attention weights are added to each layer in the graph neural network, so that the more important layers in the user feature representation get better representation.

Attention mechanism : It is a special structure embedded in the machine learning model, which is used to automatically learn and calculate the contribution of input data to output data.

related work

Current recommendation algorithm

Recommendation Algorithm Based on Matrix Factorization
Recommendation Algorithm Based on Deep Learning
Recommendation Algorithm Based on Graph Neural Network

Recommendation Algorithm Based on Matrix Factorization

Most of the traditional recommendation algorithms use matrix factorization (MF) to map users and items to n-dimensional space, and use the mapping of users and items to calculate user preferences for items, that is, to perform rating prediction tasks.
The most successful method of MF application is the LFM algorithm, which uses matrix factorization (MF) and introduces bias terms at the same time, which greatly improves the prediction accuracy.

LFM algorithm : first classify all the items, and then recommend items in this category to the user according to the user's interest classification.

Bias term : Bias unit, also known as bias term or intercept term in some materials, is actually the intercept of the function, which is consistent with the meaning of b in the linear equation y=wx+b. In y=wx+b, b represents the intercept of the function on the y-axis, which controls the distance of the function from the origin. In fact, the bias unit in the neural network has a similar effect. Therefore, the parameters of the neural network can also be expressed as: (W, b), where W represents the parameter matrix, and b represents the bias term or intercept term.

On the basis of LFM, the SVD++ algorithm adds the calculation of the implicit feedback vector of the user's historical rating products to correct the prediction results.
The PMF algorithm introduces the probability distribution function into MF, and provides a new idea of matrix decomposition.
The RMR and TopicMF algorithms use the topic model LDA to extract text features, and use the score data to further improve the prediction accuracy.
Aiming at the noise problem caused by user data privacy protection in the recommendation system, the PGMF algorithm introduces the genetic algorithm into matrix decomposition, which not only provides good protection for user data, but also reduces the noise problem and ensures the accuracy of the recommendation results.

Recommendation Algorithm Based on Deep Learning

ConvMF uses CNN to extract user item preferences from reviews. Compared with the matrix decomposition method, the performance of the algorithm has been greatly improved.
DeepCoNN uses a parallel CNN structure to simultaneously extract user item preferences
LCPMF combines LDA topic model in matrix factorization with CNN to comprehensively utilize topic and deep semantic information of review documents
DIN introduces the attention mechanism to learn the expression of user items, and has achieved good results
MAGN adopts a multi-head attention mechanism, adding new attention to the attention mechanism to obtain the influence of different friends of the user on the user in different aspects
Wide & Deep considers low-order features and high-order features at the same time, improves the effect, and uses multi-threading to improve processing efficiency
CapIPTV introduces the capsule network, uses the dynamic routing mechanism and clustering characteristics of the capsule network, and uses the attention mechanism to obtain different interest preferences of different users
After CNN, using new deep learning methods such as RNN, LSTM, and Transformer, new recommendation algorithms such as JNTM, RNN, and BST have emerged.

Recommendation Algorithm Based on Graph Neural Network

The NGCF model introduces GCN into the recommendation algorithm, models the high-order connectivity of user items and gives recommendations accordingly
The KGCN-PN model sorts and samples the neighbor entities of the entities in the graph, uses GCN to fuse information along the relationship path, and obtains the entity feature vector
The KGAT algorithm integrates the attention mechanism into the graph neural network and has achieved good results
The BGANR algorithm utilizes the attention mechanism while adding bias to better capture the high-order connectivity between nodes
On the basis of GCN, NIA-GCN further considers the mutual interaction between neighbor nodes, and can effectively aggregate the neighbor node information at each depth.
The GCN-ONCF model designs GCN as an encoder, uses the outer product operation to convert the encoding vector into a two-dimensional feature matrix, and realizes the convolution matrix decomposition through the convolutional self-decoder
Based on NGCF, LightGCN simplifies the use of redundant parts in bipartite graph GCN, thereby improving model efficiency and performance

Innovation

Although GCN can extract the relationship between users and items very well, most GCN-based recommendation algorithms do not take into account the advantages of deep learning in text and other information processing

This paper first combines the attention mechanism
At the same time, use the advantages of GCN to extract the relationship between users and items
Then use the advantages of deep learning to extract the interest preferences contained in user item comments
Finally, the performance of the model is improved by means of feature fusion

model design

Problem Statement and Related Definitions

Given a dataset D containing X samples, each sample (u, i, rui) represents that user u wrote a comment rui on item i.

The core task of this article is to train a model

Learn the relationship between user u and product i based on the interaction between all users and all products (excluding the interaction between user u and product i)
Learn the general preference representation of user u and product i based on user u’s review set (user u’s comments except rui) and product i’s review set (commodity i’s comments except rui)
Combining the two features to represent the prediction of user u's rating Pui for product i

Use the same method to get user u’s rating set Pu for all products, and recommend the top K items to user u according to the rating set. The final task is to make the recommendation set closer to user u’s future purchase behavior.

LGDL Model Framework

The embedding layer is responsible for modeling products and users by using product-user interaction information and product and user comment text.
forward propagation layer

1. The interactive information modeling expressions e(0)i1 and e(0)u3 obtained by the embedding layer are passed to two parallel GCNs to extract the high-level information of products and users, where the superscript 0 represents the initial embedding, that is, the previous Express to the 0th hop in the propagation layer, and then obtain the final expressions e'i1 and e'u3 of the relationship between the final product and the user through the self-attention mechanism
2. The product and user comment text information obtained by the embedding layer is modeled and expressed bi1 and bu3 into two parallel deep learning networks (including convolutional layer, maximum pooling layer, dropout and fully connected layer) to extract general preferences of products and users Information, get the final expression b'i1 and b'u3 of the general preference of the final product and the user

The features of the two expressions are fused in the rating prediction layer to obtain the final product and user’s expressions I1 and U3, and calculate the user u3’s predicted score for product i1, and form a set Pu according to the user u3’s predicted scores for different products. According to the The prediction score sorting of the set gives the top K recommended product contents for user u3

embedding layer

The embedding layer is responsible for inputting user product ID and user product review information into the model

ID feature embedding

The ID embedding vectors of all users form a set
The ID embedding vectors of all products form a set

The ID embedding vectors of the product and the user are both in the initial state, and the embedding is further refined by propagating in the forward propagation layer, so that the ID embedding vector can better express its internal relationship

Review text feature embedding

bu and bi contain prior knowledge of comment text information, and feature extraction is performed on it through deep learning in the forward propagation layer to extract general preferences of users and products

forward propagation layer

The forward propagation layer is divided into two parallel frameworks to extract the relationship between users and products and general preferences

Relationship Extraction

Assuming that the bipartite graph formed by all known associations is G, a method similar to LightGCN is used to extract the associations between users and products

Preference Feature Extraction

The feature extraction for the general preference of the user ui first undergoes a convolution operation

Max pooling (max-pooling) operation to further extract features

input fully connected layer

Score Prediction Layer

After the forward propagation layer, the relationship between user ui and product ij can be obtained

Model optimization

Adopt the widely used BPR loss in recommender systems

The model uses the stochastic gradient descent method to train the model network, and uses the Adam optimizer to optimize the model parameters and minimize the loss function

Commodity Recommendation Algorithm Based on Graph Neural Network and Deep Learning

Traditional practice