Guide: Graph neural network, a hot word in science and technology network! ! !

Research Background

Traditional deep learning methods have been applied to extract features of Euclidean space data and have achieved great success, but the data in many practical application scenarios is generated from non-Euclidean space, and traditional deep learning methods are not effective in processing non-Euclidean space data. performance is still unsatisfactory. For example, in e-commerce, a graph-based learning system can use the interaction between users and products to make very accurate recommendations, but the complexity of graphs makes existing deep learning algorithms face difficulties in processing huge challenge. This is because the graph is irregular, each graph has an unordered node of variable size, and each node in the graph has a different number of adjacent nodes, causing some important operations (such as convolution) to be in the image is easy to compute, but is no longer suitable for direct use in graphs. Furthermore, a core assumption of existing deep learning algorithms is that data samples are independent of each other. However, this is not the case for graphs, each data sample (node) in the graph will have edges related to other real data samples (nodes) in the graph, and this information can be used to capture interdependencies between instances.

basic concept

1. What is a graph

A graph can generally be represented by an ordered two-tuple G=(V,E), where V is called the top set, E is called the edge set, the elements of the top set are called vertices (or nodes), and the elements of the edge set are called called edges. The degree of a node is the number of nodes directly adjacent to it. As shown in Figure 1, according to the mutual influence relationship between nodes, graphs can be divided into the following three categories: directed graphs, undirected graphs, and weighted graphs.

Figure 1 Three classifications of graphs

Graphs can be used to represent relationships between objects, such as chemical molecules, communication networks, social networks, etc. Different from the data format of [features, labels] in traditional machine learning, the data represented by graphs is called graph data, which not only includes the characteristics of each node, but also includes the interaction relationship between each node.

2. Graph neural network

Generally speaking, for a classification or fitting task, a neural network that can use graph data as input is called a graph neural network. In the graph neural network, according to the different needs of specific tasks, the entire graph can be used as a data sample, or each node in the graph can be used as a data sample. For example: in Figure 2, the entire molecular formula is used as a sample, and a graph neural network classifier is trained to predict the sample label of the unknown molecular formula; Interrelationships between nodes, training a graph neural network to predict labels for unknown nodes.

Figure 2 Model at the graph level

Figure 3 Node-level model

Implementation

The modeling idea of the graph neural network comes from the convolutional neural network, and its common point is that not only its own features but also the features of its neighborhood are considered and aggregated in the process of feature extraction, so as to achieve efficient feature extraction. In general, there are two approaches to modeling graph neural networks: spatial-domain graph convolution-based methods and spectral-domain graph convolution-based methods.

1. Spatial map convolution

Figure 4 Spatial map convolution

The core idea of spatial graph convolution is to aggregate domain information during the feature extraction process, and different types of graph neural networks are derived according to different aggregation methods. In NN4G (Neural Networks for Graph), each node directly adds the features of its adjacent nodes to its own features after matrix transformation, so as to obtain the graph network of the next layer. Each node in NN4G only considers the nodes directly adjacent to it (distance 1), while in DCNN (Diffusion-Convolution Neural Network), each node considers nodes with distances 1, 2, ... K at the same time . In NN4G and DCNN, each node is equally considered to its adjacent nodes (that is, given the same weight in the aggregation process), and in MoNET (Mixture Model Networks), the weight between two nodes is determined by these two The degree of the node is determined, so that the information of the graph structure can be more fully utilized. In GAT (Graph Attention Networks), the attention mechanism is introduced, and the weight between two nodes is generated by the characteristics of these two nodes. GIN (Graph Isomorphism Network) unifies the above-mentioned graph neural network based on spatial graph convolution, which means that the core idea of spatial graph convolution is the aggregation of the characteristics of each node and the characteristics of its adjacent nodes, and proposes the design graph neural network needs A few guidelines to follow.

1) The weight of the characteristics of its own node needs to be different from the weight of the characteristics of adjacent nodes, so as to distinguish itself from its neighbors;

2) In the aggregation process, it is necessary to sum the characteristics of all domain nodes, rather than seeking the maximum or averaging;

3) After aggregation, a multi-layer perceptron needs to be used for feature extraction instead of a single-layer fully connected network.

2. Spectral Domain Graph Convolution

Graph Convolutional Neural Network GCN (Graph Convolution Network) performs Fourier changes on the characteristics of the nodes and the designed graph filter, and after multiplying the two, then performs the inverse Fourier transform to realize the convolutional neural network. The convolution operation of , which provides strong theoretical support for the effectiveness of graph neural networks. Although the proof process of this theory is very complicated, the form of GCN is very simple. It only needs to multiply a fixed graph filter (this filter is only related to the structure of the graph) after making matrix changes to each node, and the graph can be realized. convolution operation.

Figure 5 Spectral Domain Graph Convolution

Summarize

Graph Neural Network (GNN) has a wide range of applications. It can process data with explicit association structures, such as drug molecules and circuit networks, as well as data with implicit association structures, such as images and texts. In recent years, GNN has been used to solve problems in various industries, such as molecular fingerprint recognition in the field of biochemistry, drug molecule design, disease classification, etc., traffic demand prediction, road speed prediction in the transportation field, and computer image processing. Target detection, visual reasoning, etc. in the field, entity relationship extraction, relationship reasoning, etc. in the natural language processing field.

Related Papers

1. Micheli A. Neural network for graphs: A contextual constructive approach. IEEE Transactions on Neural Networks, 2009, 20(3): 498-511.

2. Atwood J, Towsley D. Diffusion-convolutional neural networks. Advances in neural information processing systems, 2016.

3. Li Y, Yu R, Shahabi C, et al. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926, 2017.

4. Velicković P, Cucurull G, Casanova A, et al. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.

5. Monti, Federico, et al. "Geometric deep learning on graphs and manifolds using mixture model cnns." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

6. Wu Z, Pan S, Chen F, et al. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 2020, 32(1): 4-24.

7. Liu Zhongyu, Li Yanlin, Zhou Yang, Deep and Simple Graph Neural Network: GNN Principle Analysis, Mechanical Industry Press, 2019

Thank you for your love for the article, scan the code and add micro

Get artificial intelligence learning materials for free

If you have questions about [technical answers, learning routes, thesis guidance, projects, competitions, etc.], you can contact Tang Yudi, a doctor of computer science, to answer your questions

AI learning package, including:

① Artificial intelligence courses and projects [including courseware source code]

②Summary of artificial intelligence must-read high-quality books and e-books

An article to understand the graph neural network ()