Pooling operation of graph neural network

The graph neural network has two levels of tasks: one is the graph-level and the other is the node-level. The graph-level task is to classify or regress the entire graph (such as molecular classification), and the node-level is Perform classification regression on the nodes in the graph (traffic network road flow prediction). For tasks at the graph level, we need to aggregate the global information of the graph (including all nodes and all edges), and aggregate the global information to obtain a vector that can represent the entire graph, and then perform classification and regression operations on this vector.

How to represent global information as a vector? In image classification and regression tasks, we have pooling operations, and in graph data, we also introduce pooling operations.

table of Contents

1. Global pooling

1.1 readout

1.2 Global Virtual Node

2. Hierarchical pooling

2.1 Pooling mechanism based on graph collapse

2.2 Pooling mechanism based on TopK

2.3 Pooling mechanism based on edge shrinkage

references

1. Global pooling

1.1 readout

Readout operation (readout) [1] The simplest pooling operation, its operating formula is:

$\mathbf{y}=R({\mathbf{h}{_{i}}^{(k)}|\forall v_{i}\in V})$

Which $R$ can be an $max,sum,mean$ operation, that is, readout directly calculates the maximum value of all nodes in the graph, sums, and averages, and uses the obtained value as the output of the graph.

1.2 Global Virtual Node

The global virtual node [2] is to introduce a virtual node, which is connected to all nodes in the graph, and also participates in operations such as convolution of the entire graph. Finally, the hidden feature of the virtual node is the feature of the entire graph. The schematic diagram is as follows:

Introduce a global node $e_{0}$ , the original node in the figure is $e_ {1}, e_ {2}, e_ {3}$ .

2. Hierarchical pooling

Hierarchical pooling passes K rounds of message transmission, and nodes are more able to extract global information.

2.1 Pooling mechanism based on graph collapse

Graph collapse is to divide the nodes in the graph into several node clusters, and each node cluster is used as the node of the next graph collapse.

Currently DIFFOOL[3] and EigenPooL[4] are two graph neural network algorithms based on graph collapse

2.2 Pooling mechanism based on TopK

TopK [5] has a different pooling mechanism from the graph collapse above. Graph collapse is a process of continuously aggregating graph nodes into clusters, while TopK is a process of continuously discarding nodes, specifically setting a hyperparameter $k$ , where $k\in (0,1)$ , Then learn the value representing the $with$ importance of the node and sort the nodes in descending order according to the importance, and select the most important $kN$ node. The formula is as follows:

$i=top\_rank(z,kN)$

$X{'}=X_{i,:}$

$A{'}=A_{i,j}$

$X{'},A{'}$ Represents a slicing operation, which is to leave $kN$ a node and the edge connected to it

How to choose $with$ is also an innovative place. In the paper, the author introduces a global basis vector $p$ , and uses the projection of the node vector on the vector as the importance:

$z=\frac{Xp}{\left \| p \right \|}$

Among them $X$ is the node feature vector. The model structure of TopK given in the paper is:

The author also adds a readout operation after each layer of pooling, and sums up in the last layer, which is helpful for the information fusion even if there are nodes.

2.3 Pooling mechanism based on edge shrinkage

This method iteratively merges the nodes of each edge pair by pair and forms a new node, while retaining the connection relationship of the two nodes before the merge to the new node. EdgePool [6] is a pooled graph network based on edge shrinkage. Next, introduce its related algorithms:

First calculate a raw score for each edge:

$r_{ij}={\mathbf{w}}^{T}[h_{i}||h_{j}]+b$

As shown in the figure below (left), the raw scores of the edges calculated based on the four points A, B, C, and D

Then calculate the score of each edge according to the in-degree:

$r_{i}=e^{r_{ij}}$

The node score is the sum of the edge scores

As shown in the figure below (middle left), the value of D is:, the value of $e^{0.5}=1.6$ C is $e ^ {0.5} + e ^ 1 + e ^ 3 = 1.6 + 2.7 + 20 = 24.5$ .

Then calculate the normalization of the value of the edge to its in-degree node

As shown in the figure below (middle right), the value of the edge of C->D is: 1.6/1.5=1

Finally, the two unselected nodes with the largest scores are selected for the shrinking operation.

The node update method after shrinking is:

$h_{ij}=s(h_i+h_j)$

$s$ Can be summation, average value, full connection and other operations.

references

[1] Gilmer J, Schoenholz S S, Riley P F, et al. Neural message passing for quantum chemistry[J]. arXiv preprint arXiv:1704.01212, 2017.

[2] Pham T, Tran T, Dam H, et al. Graph classification via deep learning with virtual nodes[J]. arXiv preprint arXiv:1708.04357, 2017.

[3] Ying Z, You J, Morris C, et al. Hierarchical graph representation learning with differentiable pooling[C]//Advances in neural information processing systems. 2018: 4800-4810.

[4] Ma Y, Wang S, Aggarwal C C, et al. Graph convolutional networks with eigenpooling[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019: 723-731.

[5] Cangea C, Veličković P, Jovanović N, et al. Towards sparse hierarchical graph classifiers[J]. arXiv preprint arXiv:1811.01287, 2018.

[6] Diehl F. Edge contraction pooling for graph neural networks[J]. arXiv preprint arXiv:1905.10990, 2019.

[7] Liu Zhongyu, Li Yanlin, Zhou Yang, "In-depth explanation of the graph neural network: GNN principle analysis"