Deep Learning and Graph Neural Networks
In recent years, artificial intelligence and deep learning have made great progress in various fields. From the field of Computer Vision (Computer Vision), which was the first to set off this wave of deep learning, to the fields of Natural Language Processing (Natural Language Processing) and Data Mining (Data Mining), which have also attracted much attention, to various specialized fields, such as bioinformatics (Figure 1), game games (Figure 2), etc., artificial intelligence is increasingly showing its edge.
The biggest driving force for such progress in artificial intelligence and deep learning can be summarized as the following two points [1]:
- Enhanced data availability. With the development of the digital society and more and more digital activities (such as social networking, picture sharing, etc.), the scale of digital data available for use is also increasing. The data base is provided.
- Computational scale growth. With the emergence of neural networks, the computing scale of intelligent algorithms can be fully expanded to cope with the growth of data scale. Figure 3 points out that as the amount of data increases, the performance of traditional algorithms and small-scale neural networks will tend to be saturated, and we need to rely on larger-scale neural networks so that the algorithm can benefit more from the data and achieve better results. performance.
Looking at the current application of artificial intelligence from the perspective of data, traditional grid data (images) and sequence data have a series of mature neural networks that can model them, such as CNN for images, RNN for time series, and for other A widely existing data type: graph (Graph), effective network structure also needs to be proposed and formed into a series, so as to solve graph-related problems. Examples of graphs include: social networks, knowledge graphs, space-time observations, etc., covering all aspects of the real world. Excellent graph neural network can effectively analyze and apply graphs, and the potential benefits it can bring also make the research of graph neural network a hot spot in the field of artificial intelligence research in recent years.
Introduction to Graph Neural Networks
Through the previous section, the author has pointed out that the graph neural network is a series of neural networks applied to graph-structured data. In this section, the author hopes to have a conceptual understanding of the basic structure and operation (module) of the graph neural network before introducing the specific graph neural network.
A mainstream concept that is analogous to convolutional neural networks believes that the basic operations of graph neural networks include graph filtering (Graph Filtering) and graph pooling (Graph Pooling), which correspond to convolution and pooling operations in convolutional neural networks. Referring to [2], the graph filtering operation and graph pooling operation can be represented by Figure 4 and Figure 5, respectively.
Graph filtering operations are used to extract features from graph-structured data. In view of the rapid development of neural networks in recent years, graph filtering operations have also been extended to include convolution operators, loop operators, attention operators, jump connection operators and other basic operators for graph-structured data. The author will introduce separately.
As a supplement, when the graph structure is too complex or huge, and the above graph operations are difficult to meet the constraints of computational complexity and hardware, it is necessary to introduce graph sampling (Graph Sampling) operations to simplify the overall calculation. Common graph operations include Node-wise Sampling, Layer-wise Sampling, and Subgraph-wise Sampling.
Graph pooling is to downsample the features extracted by graph filtering operations to meet the (global) feature requirements of some specific tasks. Graph pooling operations can be classified into direct (global) pooling and hierarchical pooling, which is also a distinction made in response to complex or large-scale graphs, that is, hierarchical pooling may be required to perform these complex graphs processing so as not to lose the rich hierarchical information contained in these graphs.
In summary, the basic structure of the graph neural network can be expressed as follows [3]:
references
[1] Andrew Ng, “Machine Learning Yearning,” 2017. [Online]. Available: https://deeplearning-ai.github.io/machine-learning-yearning-cn/
[2] Yao Ma and Jiliang Tang, “Deep Learning on Graphs,” Cambridge University Press, 2021. [Online]. Available: https://cse.msu.edu/~mayao4/dlg_book/
[3] Jie Zhou et al., “Graph neural networks: A review of methods and applications,” arXiv:1812.08434 [cs, stat], Oct. 2021, Accessed: Nov. 01, 2021. [Online]. Available: http://arxiv.org/abs/1812.08434