What I want to share with you today is the graph convolutional neural network. With the development of artificial intelligence, many people have heard of concepts such as machine learning, deep learning, and convolutional neural networks. But the graph convolutional neural network is not mentioned by many people. So what is a graph convolutional neural network? To put it simply, the object of its research is graph data (Graph), and the research model is a convolutional neural network.
Why are there graph convolutional neural networks
Since 2012, deep learning has achieved great success in computer vision and natural language processing. How is it better than traditional methods?
Suppose there is a picture to be classified. Traditional methods need to manually extract some features, such as texture, color, or some more advanced features. Then put these features into a classifier like random forest, give an output label, tell it which category it is. And deep learning is to input a picture, pass through the neural network, and directly output a label. Feature extraction and classification are in place in one step, avoiding manual feature extraction or manual rules, and automatically extracting features from raw data is a kind of end-to-end (end-to-end) learning. Compared with traditional methods, deep learning can learn more efficient features and patterns.
The convolutional neural network is very good, but the object of its research is still limited to the data of Euclidean domains. What is Euclidean data? The most notable feature of Euclidean data is the regular spatial structure, such as pictures are regular square grids, and speech is a regular one-dimensional sequence. And these data structures can be represented by one-dimensional and two-dimensional matrices, and convolutional neural networks are very efficient to process.
However, a lot of data in our real life does not have a regular spatial structure, which is called Non Euclidean data. For example, graphs abstracted from recommendation systems, electronic transactions, computational geometry, brain signals, and molecular structures. The connection of each node of these graph structures is different. Some nodes have three connections, and some nodes have two connections, which is an irregular data structure.
The following combines two typical business scenarios to illustrate what a graph is:
Social networks are well suited for graph data
The graph above depicts each node in the social network and the relationship between them. User A, user B, and posts are all nodes. The relationship between users is follow, and the relationship between users and posts may be posting or reposting. Through such a map, it is possible to analyze who and what the user is interested in, and further realize the recommendation mechanism.
Graphs in e-commerce scenarios
In e-commerce, the key nodes that we can first think of are users, transactions and commodities. For example, the nodes associated with users will have registration addresses, harvest addresses, etc.; trades will be associated with commodities, delivery addresses, transaction IPs, etc., and commodities will be associated with categories, etc. The relationship between these nodes, for example, users can not only purchase products through transactions, but also rate products. We can use such graph data to do two things, one is recommendation and the other is anti-fraud.
Through the above two examples, it can be clearly felt that graphs have two basic characteristics:
One is that each node has its own characteristic information. For example, for the above picture, we establish a risk control rule, which depends on whether the user's registration address, IP address, and transaction delivery address are the same. If these feature information do not match, the system will determine that the user has a certain degree of fraud risk. This is the application of graph node feature information.
The second is that each node in the graph also has structural information. If there are many trading nodes connected to a certain IP node in a certain period of time, that is to say, there are many edges extending from a certain IP node, then the risk control system will determine that this IP address is at risk. This is the application of information about the graph node structure.
In general, in the graph data, we need to consider both the feature information and the structural information of the nodes. If we rely on manual rules to extract, many hidden and complex patterns will be lost. Is there a way to automatically and simultaneously How about learning the feature information and structural information of the graph? ——Graph Convolutional Neural Network
What is a Graph Convolutional Neural Network
Graph Convolutional Network (Graph Convolutional Network) is a method that can perform deep learning on graph data.
How to understand the graph convolution algorithm? We look at the animation in three steps to understand (note that different colors represent different weights):
Step 1: Send (send) Each node sends its own characteristic information to neighbor nodes after transformation. This step is to extract and transform the feature information of the nodes.
Step 2: Receive (receive) Each node gathers the feature information of its neighbor nodes. This step is to fuse the local structure information of the nodes.
The third step: Transform (transform) gathers the previous information and performs nonlinear transformation to increase the expressive ability of the model.
Graph convolutional neural networks have the following properties of convolutional neural networks:
1. Local parameter sharing, the operator is applicable to each node (the circle represents the operator), shared everywhere.
2. The receptive field is proportional to the number of layers. At the beginning, each node contains the information of its direct neighbors, and when calculating the second layer, the information of its neighbors’ neighbors can be included, so that more and more information is involved in the calculation. full. The more layers, the wider the receptive field, and the more information involved in the calculation.
Let's look at the model framework of GCN. The input is a graph, which is calculated and transformed layer by layer, and finally outputs a graph.
The GCN model also has three properties of deep learning:
1. Hierarchical structure (features are extracted layer by layer, and each layer is more abstract and advanced);
2. Non-linear transformation (increase the expressive ability of the model);
3. End-to-end training (no need to define any rules, just mark the nodes of the graph, let the model learn by itself, and integrate feature information and structural information.)
Four characteristics of GCN:
1. GCN is a natural extension of the convolutional neural network on the graph domain.
2. It can simultaneously learn end-to-end node feature information and structural information, and is currently the best choice for graph data learning tasks.
3. Graph convolution has a wide range of applicability and is suitable for nodes and graphs of any topology.
4. In tasks such as node classification and edge prediction, the effect on public data sets is far superior to other methods.
How We Use Graph Convolutional Neural Networks
Let's share an experiment we did in a practical application scenario:
The experimental input is a graph data composed of verification data, and the nodes are verification events and event-related attribute nodes. Such as IP, DeviceID, UA and other nodes. (We used a total of 30 days of verification data, and every two hours of data constitutes a graph, a total of 360 graphs.)
The output of the experiment is human-machine classification of event nodes, normal or abnormal.
Experiment Details
Network structure: GCN(128)->GCN(64)->GCN(64)->Linear(2)
Training: Adam optimizer, lr=0.001
Reference benchmark: Based on GBDT, which can only learn feature information, grid_search searches for hyperparameters. GBDT is currently the most popular shallow classifier.
We use the data of the first day for training, and the prediction results for 30 days are as follows:
The accuracy rate decay of the GCN model is relatively small, while the decay of GBDT is serious. It can be seen that the human-machine discrimination effect of the GCN model is better, and the robustness is good.
7d Visualize the evaluation effect, (train the model with the data of the first day, observe its prediction effect and the tsne visualization result output by the last layer on the seventh day). As can be seen from the above figure, the interface of GCN's discrimination of samples is still obvious on the seventh day, but the interface of GBDT's discrimination of samples is already very vague. In summary, the structural information learned by GCN is not only effective in human-machine discrimination, but also has better robustness.
Organize learning materials:
Graph Neural Network Commercial Application - Package Matching [with PPT and video materials], WeChat search experience, or search WeChat ID: geetest_jy, reply "PPT" in the background to obtain
NeurIPS 2019 super complete collection of GNN papers, with original PDF download, WeChat search extreme experience, or search WeChat ID: geetest_jy, reply "NIPS2019" in the background to get
William L and Tang Jian AAAI2019 "Graph Representation Learning" report, WeChat search experience, or search WeChat ID: geetest_jy, backstage reply "AAAI2019" to get
Shared bicycle traffic forecast based on graph convolutional neural network, WeChat search experience, or search WeChat ID: geetest_jy, the background replies "New York" and "Chicago" respectively to obtain the download address of the data set
write at the end
Due to the limited time, many questions have been scratched, and there are still many interesting things about GCN. We have set up a column "Graph Learning" to share with you a more comprehensive graph learning algorithm.
For a long time, we have considered ourselves a technology-driven company and also called ourselves an AI company. AI companies are not so big, in fact, sometimes it is difficult, because many technologies are immature and there are no ready-made ones available. When doing company business, you will encounter some practical difficulties and step on many pitfalls. Of course, you will also gain some insights and experience. We think that these are another kind of value created by enterprises and should be well utilized.
"You have one mind, I have one mind, and after exchange, we both have two minds" This is the meaning of sharing. Therefore, we will organize a series of dry goods sharing columns in the later stage, hoping to share the knowledge summarized, learned and created in practical applications with everyone. Of course, people are also very welcome to discuss, communicate, and make progress together. This is the feedback we are most looking forward to in doing this.
Graph learning column:
Extreme experience official account article recommendation:
Graph convolution practice - text classification
Application of Graph Convolution to Skeleton-Based Action Recognition
Practice of Group Mining Algorithm Based on Graph
Emotion Recognition with Graph Convolutional Neural Networks
Discuss the writing ideas of GCN from the source
Various convolutions defined on graphs
Figure open source deep learning library summary