This article will share with you the paper "Spatial Heterophily Aware Graph Neural Networks" published by Xiao Congxi and Zhou Jingbo, developers of the Paddle community, at KDD2023, the top data mining conference.
Xiao Congxi
Xiao Congxi is a research intern in the Business Intelligence Laboratory of Baidu Research Institute and a doctoral candidate at the University of Science and Technology of China. He is mainly engaged in research work related to spatio-temporal data mining and deep learning of graphs. Completed several papers based on flying paddles, and published them in top computer academic conferences such as KDD and AAAI.
Zhou Jingbo
Zhou Jingbo, senior technical expert (senior PPDE) of Paddle Developer, is currently a senior researcher in the Business Intelligence Laboratory of Baidu Research Institute. He is mainly engaged in research and application work related to data mining and machine learning, including spatiotemporal big data, deep geometric learning, and knowledge graphs. And AI-assisted drug design, PaddleSpatial technical director, based on the flying paddle to complete a number of papers, published in KDD, AAAI, TKDE and other top computer conferences and journals.
background introduction
In recent years, Graph Neural Networks (GNNs) have been widely used in smart city computing. Considering that a city is a complex system and there are various connections between urban entities, many research works model the city as an urban graph (Urban Graph), where the nodes on the graph represent some kind of urban entities, and the edges represent the connections between entities. A certain association, and adopt graph neural network to learn the city graph to solve various downstream tasks in the city.
However, unlike general graphs, urban graphs often have spatial heterophily (Spatial Heterophily), which limits the performance of general graph neural networks. First of all, Heterophily and Homophily are two relative concepts. The general GNN model assumes that there is a good assortment in the graph data, and that adjacent nodes have similar characteristics. However, due to the complex relationship between urban entities with different functions, urban graphs often have heterogeneity, that is, connected nodes may not be similar. For example, residential areas and workplaces used to have a frequent population movement relationship, but it is clear that there are huge differences between the two areas. General homophilic GNNs tend to produce similar representations for neighboring nodes, and may ignore important difference information, limiting their effectiveness on heterogeneous city graphs.
In this paper, we further found that the heterogeneity of neighbors in urban graphs usually shows a certain degree of spatial diversity, which we call spatial heterophily. For a general disassignment graph, the neighbors and the central node have differences; on the urban graph, the neighbors located in different geographical locations have different distributions of the central node's difference, not uniform, that is, the difference has spatial diversity (Spatial Diversity). This paper designs a spatial diversity score index (Spatial Diversity Score) to describe the spatial heterogeneity of urban maps. As shown in Figure 1(a), the city map may obtain a higher score, indicating that there is spatial diversity in the distribution of neighbor differences on the map, that is, spatial heteroassortment.
Figure 1 Spatial heteroassortment analysis
Even though some researchers have begun to study the problem of graph heterogeneity, the existing heterogeneous graph neural network (Heterophilic GNNs) mainly studies heterogeneous graphs with limited neighbor differences, such as assuming that there are only two types of nodes on the graph. , but cannot consider the spatial diversity of the distribution of neighbor differences on the city map. As shown in Figure 1(b), we experimentally compare the performance of different GNN models on a series of artificially synthesized graphs. When the spatial disparity of the graph is gradually added (the score gradually increases), the existing disassortative graph neural network cannot maintain excellent performance. Therefore, it is very meaningful to design a graph neural network that can solve spatial heterogeneity and better perform representation learning on city maps.
To solve this problem, this paper proposes a Spatial Heterophily Aware Graph Neural Network (SHGNN). The model structure is shown in Figure 2. The design of the model is inspired by the first law of geography "everything is related, but things that are close are more closely related", that is, in cities, we can observe that urban entities with similar spatial locations usually have similar characteristics. Based on this feature, the core idea of this method is to divide the neighborhood according to the spatial position, and group the neighbors with similar space into a group, so that the neighbors in the group have similar difference distributions with the central node, so as to alleviate the problem of neighbors in the group. Anisogamous diversity. On this basis, we design a graph learning algorithm that can simultaneously model differential information, process each group separately, and solve the spatial heterogeneity of urban graphs in a divide-and-conquer manner.
In this work, we realized the model building and training based on flying paddles. In terms of input data, this paper uses the paddle graph learning framework Paddle Graph Learning (PGL) to efficiently construct and store city graphs, including information such as the connection relationship between nodes, the spatial coordinates of nodes, and the spatial distance between nodes. In terms of models, this paper first combines the subgraph extraction interface and message passing mechanism of PGL to conveniently realize the operation of message aggregation for neighbors in different spatial locations; then, based on the tensor matrix operation of flying paddles, the city map is realized. On the interaction of common information and difference information, the representation learning of city graph is enhanced. After end-to-end training of the model based on the fly paddle dynamic graph framework, the proposed method shows good performance in different downstream tasks.
method framework
Figure 2. Graph neural network for spatial disparity perception
The spatial heterogeneity-aware graph neural network proposed in this paper is mainly composed of two modules, namely Rotation-Scaling Spatial Aggregation and Heterophily-Sensitive Spatial Interaction. .
Rotate-Scale Spatial Aware Neighborhood Aggregation
The rotation-stretching space-aware neighborhood aggregation first divides the neighbor nodes, and assigns the neighbors with similar positions to the same spatial group (Spatial Group), so that the neighbors in the group have similar difference distributions to the central node, so as to alleviate the difference distribution. diversity. Next, we perform feature aggregation on the neighbor nodes of each spatial group separately. The neighborhood division and group aggregation are the basis for solving spatial heteroassortment in a divide-and-conquer manner.
Rotate-scale dual-view space division
Figure 3 Schematic diagram of rotation-stretching dual-view space division
As shown in Figure 3 (a) and (b), first divide the geographic space around each central node from the direction dimension (Direction View) and distance dimension (Distance View), and generate multiple disjoint subspaces, And group each neighbor node according to the subspace it is in. Among them, we divide the geographical space into several sectors (Sectors) with different directions in the direction dimension, and divide the space into several rings (Rings) with different distances in the distance dimension. We realized the above space division function based on the flying paddle: first, use the node feature and edge feature access API of the flying paddle PGL. Areas and rings; then, the subgraph API based on PGL.sampling can easily define each sector and each ring as a different subgraph to complete the division of neighbor nodes. The code is as follows.
Considering the following special circumstances: some neighbor nodes may be distributed on the boundary of two subspaces, and it is impossible to determine which group they belong to; we further propose a rotation-scale multi-partition strategy, which performs multiple partitions in both direction and distance dimensions, making different Complementary advantages can be played between the partitions, as shown in Figure 3(c) and (d). In terms of implementation, we define the rotation angles of multiple sets of sector boundaries and the distance intervals of multiple sets of ring boundaries through flying paddles, and call the space division function multiple times to achieve multiple space divisions. The code is shown below.
Spatial Aware Neighborhood Aggregation
After the space division is completed, feature aggregation and message passing are performed in the neighborhood. General GNNs usually use summation or averaging to aggregate neighborhood features, which will not be able to distinguish neighbors with different spatial distributions, resulting in disparate distributions with spatial diversity being mixed together, which is difficult to handle. In contrast, this method performs group-wise aggregation on the neighbors in each spatial grouping to achieve "divide and conquer" spatial heterogeneity. The aggregation process is shown in Figure 2(a). Based on the divided PGL subgraph structure, we can easily implement message passing and feature aggregation in each spatial grouping with the help of PGL's message passing method SEND-RECV. Take the neighbor aggregation in the sector under the direction dimension as an example, the code is as follows.
Spatial Interaction of Different Match Perception
On this basis, the spatial interaction module of disparity perception includes two learnable kernel functions (Kernel Function), which adaptively extract and utilize the common information from each spatial group to the central node and between spatial groups on the city map (Commonality) and difference information (Discrepancy).
common kernel function
Considering that different spatial groupings are neighbors of the central node, the use of neighborhood common knowledge (Common Knowledge) or similarity features has been widely verified to be beneficial to graph representation learning. Therefore, we first design a Commonality Kernel Function to capture the commonality information between spatial groups, and use the commonality information to enhance the representation of each group, as shown in Figure 2(b). Taking the direction dimension as an example, based on the paddle tensor calculation, we realized the measurement of the commonality between different sectors and updated the sector representation with the commonality information.
difference kernel
In addition to common knowledge, modeling the difference information of neighbor nodes is even more crucial for heterogeneous city graphs. Therefore, we designed another discrepancy kernel function (Discrepancy Kernel Function) to capture the dissimilarity between the central node and the spatial group, and between each spatial group, and similarly use the difference information to enhance the representation of each spatial group. Taking the direction dimension as an example, the code implementation is as follows.
attention gating mechanism
In various application scenarios, different urban entities on the city map may have different degrees of spatial heterogeneity. Therefore, we further implemented an attention gating mechanism (Attentive Gate) based on the flying paddle, and adaptively learned the importance of common information and difference information for node representation learning in a specific task in an end-to-end manner, so as to learn the importance of the two components Perform fusion.
Spatial Dimension Fusion
Finally, we define a learnable scale parameter through the paddle, fuse the neighborhood representations obtained under the direction dimension and distance dimension, and update the representation of the central node.
In different applications, different losses can be used to optimize the network to obtain the final representation of nodes, which can be used for node prediction tasks.
experiment
We conduct experiments on three real datasets for three city tasks, verifying the importance of accounting for spatial heterogeneity on city maps and confirming the effectiveness of our method. Compared with the general same-assignment graph neural network, dissimilar graph neural network, and spatial graph neural network, this method can achieve better performance in different downstream tasks.
Table 1 Performance comparison in three urban tasks
In addition, we have also verified the role of each part of the design of this method through a wealth of ablation experiments, including modeling spatial disparity from two spatial dimensions, using rotation-scale multiple partitions, and simultaneously capturing commonality and difference information on the map, etc. .
Figure 4 Ablation experiment
Summarize
This paper investigates the unique problem of spatial assortativeness on city maps. On the one hand, we design a metric to describe the spatial heterogeneity of urban graphs and analyze its impact on graph neural networks; on the other hand, we implement a new spatial heterogeneity perception based on the flying paddle graph learning framework The graph neural network can easily group neighbors according to spatial division, solve the problem of spatial disparity in urban graphs by divide and conquer, and achieve performance improvement in multiple urban tasks.
Relevant codes have been open sourced on the PaddleSpatial space-time computing platform. PaddleSpatial is a spatio-temporal big data computing tool and platform developed based on the deep learning framework of Baidu Flying Paddle. It integrates Baidu's leading spatiotemporal capabilities such as region segmentation, spatiotemporal transfer learning, and time series prediction, and can support the application of various spatiotemporal computing scenarios.
Paper
https://arxiv.org/abs/2306.12139
Code
https://github.com/PaddlePaddle/PaddleSpatial/tree/main/research/SHGNN