Word2vec and item2vec derived from it are the basic methods of embedding technology, but both are based on "sequence" samples (such as sentences, recommendation lists). In the Internet scenario, the graph structure is more present among data objects.
For reference materials, see the link for detailed interpretation.
Graph Embedding method you have to learn in deep learning
1. Classic Graph Embedding method-DeepWalk
Summary: DeepWalk is based on the user behavior sequence of the original user and constructs an "item correlation graph" according to the behavior sequence of each user, and then adopts a random walk method to randomly select the starting point and regenerate the item sequence.
Finally, these generated item sequences are input into the word2vec model, and finally the embedding of the items is generated.
2. Further improvement of DeepWalk-Node2vec
In 2016, Stanford University went a step further on the basis of DeepWalk. By adjusting the weights of random walks, the results of graph embedding were weighed in the homogeneity (homophily) and structural equivalence of the network.
3. Ali's Graph Embedding method EGES
In 2018, Ali announced its embedding method EGES (Enhanced Graph Embedding with Side Information) for Taobao applications. The basic idea is to introduce supplementary information on the basis of graph embedding generated by DeepWalk.
If you simply use the item-related graph generated by user behavior, you can generate embedding of items, but if you encounter newly added items or long-tail items without too much interactive information, the recommendation system will have a serious cold start problem. In order to obtain a "reasonable" initial embedding for "cold start" products, the Ali team has introduced more supplementary information to enrich the source of Embedding information, so that products without historical behavior records can obtain Embedding.
The first step in generating Graph embedding is to generate an item relationship graph. The item relationship graph can be generated through the user behavior sequence. The same attribute, the same category and other information can also be used to establish the edges between items through these similarities to generate content-based knowledge graph. The item vector generated based on the knowledge graph can be called a side information embedding vector. Of course, there can be multiple side information embedding vectors according to different types of supplementary information.
So how to fuse multiple embedding vectors of an item to form the final embedding of the item? The easiest way is to add an average pooling layer to the deep neural network to average the different embeddings. Ali has strengthened on this basis and added a weight to each embedding, as shown in Figure 7, for each type of feature. Embedding vectors are given weights a0, a1...an. The Hidden Representation layer in the figure is the layer that performs weighted average operations on different Embeddings. After the weighted average Embedding vector is obtained, it is directly input to the softmax layer, so that through the gradient back propagation, the weight of each embedding can be obtained ai( i=0...n).