[Review Paper] Summary of Graph Neural Network

The following is based on the following literature:

A Comprehensive Survey on Graph Neural Networks

Graph neural network

Graph Neural Network Based on Model Structure Classification

• Common Recurrent Graph Neural Networks and Convolutional Neural Networks
  
• Performance comparison of common convolutional neural networks
  

1. Recurrent Graph Neural Networks (ResGNNs)

Recurrent graph neural networks (ResGNNs) : Assume that a node in the graph is constantly exchanging information/messages with its neighbors until a stable equilibrium is reached

2. Convolutional Graph Neural Network

Convolutional graph neural networks (ConvGNNs) : by gathering nodes $Features\; of\; v$ itself$x_v$and neighbor features $x_u$to generate a representation of node v, where $u\in N\left(v\right)$ . Different from RecGNNs, ConvGNNs stack multiple graph convolutional layers to extract high-level node representations. Figure (a) below represents node classification, and figure (b) represents graph classification.

Spectral-based ConvGNNs (Spectral-based ConvGNNs)

Representative models : ChebNet, GCN, DGCN (Directed Graph Convolutional Network), lightGCN

Spatial-based ConvGNNs (Spatial-based ConvGNNs)

Representative models : GraphSage, GAT, LGCN, DGCNN, DGI, ClusterGCN

Comparison of Spectral Domain Graph Convolution Model and Spatial Domain Graph Convolution Model

Spatial models are preferred over spectral models due to efficiency, generality, and flexibility issues.

1. Spectral models are less efficient than spatial models : Spectral models either require eigenvector computations or process the entire graph at once. The spatial model is directly convoluted in the graph domain through information propagation, thus having stronger scalability for large graphs. Computations can be performed in a batch of nodes rather than across the entire graph.
2. Spectral models that rely on graph Fourier bases generalize poorly to new graphs : they assume a fixed graph. Any perturbation to the graph results in a change in the eigenbasis. Spatial-based models perform local graph convolutions at each node, where weights can be easily shared across different locations and structures.
3. Spectral-based models are limited to operating on undirected graphs. Space-based models handle multi-source graph inputs more flexibly : such as edge inputs [15], [27], [86], [95], [96], directed graphs [25], [72], signed graphs [97], heterogeneous graphs [98], [99], as these graph inputs can be easily incorporated into aggregation functions.

Downsampling Strategies in Graph Convolutional Models

After GNN generates node features, we can use them for the final task. However, directly using all these features is computationally challenging, thus, a downsampling strategy is required. This tactic has different names depending on the goal and the role it plays in the network, but their mechanics are very similar. :

• Pooling : Aims to reduce the size of parameters by downsampling nodes to generate smaller representations, avoiding overfitting, permutation invariance, and computational complexity issues
• Readout : Mainly used to generate graph-level representations based on node representations.
  $$h_G=mean/max/sum(h_1^{(K)},h_2^{\left(K\right)},...,h_n^{\left(K\right)}),Kis\; the\; index\; of\; the\; current\; last\; graph\; convolutional\; layer.$$

3. Graph Autoencoders (GAEs)

Graph autoencoders (GAEs) : are an unsupervised learning framework that encodes nodes/graphs into a latent vector space and reconstructs graph data from the encoded information. GAEs are used to learn network embeddings and graph generation distributions. Figure © shows GAE for network embedding.

4. Spatio-Temporal Graph Neural Networks (STGNNs)

Spatial-temporal graph neural networks (STGNNs) aim to learn hidden patterns from spatio-temporal graphs, which is becoming increasingly important in various applications, such as traffic speed prediction [72], driver maneuver prediction [ 73] and human action recognition [75]. The key idea of ​​stgnn is to consider both spatial and temporal dependencies. Figure (d) shows STGNN for spatio-temporal graph prediction.

Representative model:

GCRN、DCRNN、Structural-RNN 、ST-GCN、StemGNN

Summarize

• Recurrent Graph Neural Networks (ResGNNs): GNN, GraphESN, GGNN, SSE
• ConvGNNs based on spectral domain: ChebNet, GCN, DGCN (Directed Graph Convolutional Network), lightGCN
• Space-based ConvGNNs: GraphSage, GAT, LGCN, DGCNN, DGI, ClusterGCN
• Graph Autoencoders (GAEs): GAE, VGAE
• Spatiotemporal graph neural networks (STGNNs): GCRN, DCRNN, Structural-RNN, ST-GCN, StemGNN

Applicable models for various graph types

1. Directed graph

Applicable models : GraphSAGE, GAT

2. Undirected graph

GCN

3. Heterogeneous graph

Heterogeneous graph: refers to the existence of several different types of nodes

4. Graphs with Side Information

Graph with edge information: The edges carry additional information. There are two solutions:

• Divide this type of graph into a bipartite graph (bipartite graph: divide the vertices of a graph into two disjoint subsets, so that each edge can connect the vertices in the two sets respectively. If there is such a partition, then This graph is a bipartite graph), turning edges with information into nodes. That is: start node - edge node - end node
• For different types of edges, different weight matrices are used to propagate

5. Dynamic graph

Dynamic graphs: Representing complex networks as time-varying structures

https://arxiv.org/pdf/2005.07496.pdf
https://arxiv.org/pdf/2203.10480.pdf

6. Multidimensional graphs

A multidimensional graph refers to the fact that there are multiple relationships between nodes to form a multidimensional graph

data set

academic words

• Taking sth as an example: Take... as an example