Figure Network Spatial Convolution Description 4: PGC

论文地址：Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

1. The core idea

PGC believes that convolution can be regarded as a specific sample function (sample function) and a specific weight function (weight function) multiplied and then summed.
In the paper, this model acts on the task of action recognition based on human skeleton.

Second, traditional convolution comparison

For traditional convolution, a $K\times$The convolution operation of the$K$ convolution kernel can be considered as the following function:

where$K$ is the size of the convolution kernel (commonly 3, 5). $p\left(\right)$ is a sampling function, which takes the nodes out of the neighborhood in turn for convolution calculation. $w\left(\right)$ is a weight function, that is, a parameter for convolution operation on each node.
For example: for$3\times$Convolution kernel of$3$ ,$k=3$，$p(x,1,1)$ is the first point in the upper left corner,$w(1,1)$ Represents the coefficient in the upper left corner of the convolution kernel.

The entire formula isthe inner product ofnode featuresandconvolution kernel parameters.

Three, PGC implementation

PGC is to apply the convolution idea thought above to the graph structure. The main process is to take a suitable sampling function and a corresponding weight function .

Sampling function

The sampling function is to take out the nodes sequentially in the neighborhood. The focus is on how to construct the neighborhood of the node , that is, where the sampling function takes samples.

On the graph structure data, PGC can define the sampling function on the D-order neighbor nodes . That is, $B\left(v_i\right)=\left\{v_j\mid d\left(v_j,v_i\right)\le D\right\}$ , where$d(v_j,v_i)$ Means from$i$ node tojjThe shortest distance of$j$ node.

In the experiment, take $D=1$ , and sample one by one in the first-order neighborhood. But it can also be set to other neighborhoods.

Weight function

First divide the points in the sampled neighborhood into $K$ different classes.
li: B (vi) → {0,…, K − 1} l_i:B(v_i) \rightarrow\{0,\dots,K-1\}li​:B(vi​)→{ 0,…,K−1}

Where $l_i(\cdot )$ represents theiiClassification mapping of$i$ nodes. $B\left(v_i\right)$ DenotesiiNeighborhood nodes of$i$ nodes. {0,…, K − 1} \{0,\dots,K-1\}{ 0,…,K−1 } stands for category.

The same class shares a convolution kernel parameter. Convolution kernels between different classes do not share parameters:
$$w(v_i,v_j)=w^{\prime }(l_i(v_j))$$

表示$v_i,v_j$The convolution kernel parameter between is equal to the convolution kernel of the neighborhood class to which it belongs.

Classification strategy

($a$ ): Enter the sample frame of the skeleton. The joints of the body are painted with blue dots. $D=$The receptive field of the filter of$1$ is drawn with a red dotted circle. The red node is the center node, and the circle is its first-order neighborhood.

($b$ ):Uni-labeling classification
treats equally, only one category, similar to GNN.

($c$ ):Distance classification is classified
according to the order, the central node is 0 order, and the adjacent nodes are 1 order, which are divided into different classes.

($d$ ):Spatial configuratio classification
According to the distance from the center of the human skeleton, there are three categories. The node distance is less than the distance between the center node and the skeleton center is one type, the node distance is greater than the distance between the center node and the skeleton center is one type, and the center node is one type.

official

The final convolution formula is as follows:

${WITH}_i\left(v_j\right)$ Means jjin the neighborhood of the i node$The$ number of nodes in the category of the$j$ node (category =$v_j$), used to balance each type of node information.

Contrast thinking

1. Compared with the mean sampling of GraphSAGE, PGC defines a sampling function with stronger generalization.

2. GNN thinks that the nodes need to be sorted, GraphSAGE thinks that there is no need to sort, while PGC uses a weight function, if it is divided into only one category, it is similar to GNN, if it is considered that each node category is different, then it is similar to GraphSAGE, with stronger Generalization.

For example