Overview of Graph Neural Network for Time Series (GNN4TS)

Today I will give you a review on using graph neural networks for time series analysis. The download address of the literature is as follows:

https://arxiv.org/abs/2307.03759

overview

Time series can be said to be everywhere, such as weather forecasts, stocks, item sales, car trajectories, etc. Therefore, time series analysis is of great significance to many fields such as cloud computing, energy, and social networks. Time series data usually involves complex interaction patterns between data at different time points along the time dimension (such as lags that affect the spread of benefits) and variable dimensions (the mutual influence of adjacent sensing devices). Many time series data even exhibit spatiotemporal properties. , that is, variables measured by different sensors contain different location (spatial) relationships. Changes in local variables will be propagated to surrounding variables, such as the increase in traffic volume on adjacent roads, the spread of diseases and severe weather in neighboring areas, which is time Sequence analysis poses significant challenges. With the rapid development of graph neural network-related research, more and more graph neural network-related algorithms are used in time series modeling. By treating time points or variables as nodes and their relationships as edges, constructing models in the form of networks or graphs can effectively learn complex interaction patterns in time series, which is useful in urban transportation networks, population migration, weather forecasting, etc. It has been widely used in scenes.

Consider that time series analysis is not just about understanding the information in historical data, but also involves a variety of tasks such as prediction, classification, anomaly detection, and data interpolation. This review starts from the four task dimensions of time series prediction, classification, anomaly detection and data filling, and reviews related research on graph neural networks in the field of time series. It is worth noting that this article is the first to comprehensively summarize the application of graph neural networks in mainstream tasks of time series analysis. The article also lists other related reviews as follows. Interested friends can refer to them as needed.

Theorems and symbolic representations

Univariate time series: , is a regular sampling time. If it is an irregular sampling sequence, it can be expressed as.
Multivariate time series: , is the sampling moment. GNN-related research mainly focuses on multivariate time series. The relationship between different time series can be represented by the following space-time diagram.
Attribute graph: It consists of a node characteristic matrix and its corresponding adjacency matrix. The adjacency matrix describes the topological graph structure between N nodes, including N nodes, the edges between the nodes, and the weight of the edges.

To use GNN to analyze time series, you usually need to provide a graph structure, that is, an adjacency matrix. However, not all time series data have ready-made graph structures available, and in practice, two strategies are usually used to obtain graph structures from data: heuristic methods or direct learning methods.

Heuristic graph: This type of method discovers the graph structure from the data based on some heuristic indicators, such as spatial similarity, whether node pairs are connected, node feature similarity, determining the node's structure based on functions such as Granger causality or transfer entropy. Connection and adjacency matrices.
Graph learning-based methods: This type of method will directly use the adjacency matrix as a parameter to learn during end-to-end training of downstream tasks. Compared with heuristic graphs, this type of method can learn more complex or informative graphs.

Graph neural networks for different time series analysis tasks

The article summarizes the related research of GNN in the four key task dimensions of time series prediction, time series anomaly detection, classification and anomaly detection. These methods share a common pipeline, which uses space-time graphs to model time series variables to achieve Representation of a single sequence as shown below. Among them, time series data representation is performed through spatiotemporal graph (STGNN), which can transform the time series classification task into a graph classification or node classification task.

The space-time diagram in the above pipeline consists of two key modules: spatial module and temporal module. Among them, the spatial module is used to model the interaction between different time series (nodes), and spectral GNN, spatial GNN, or a combination of the two methods are often used. Spectral GNN is based on spectrogram theory and uses graph offset operators (such as graph Laplacian) to capture node relationships. Spatial GNN simplifies spectral GNN and directly filters the neighbors of each node through filters. The hybrid method is a set The advantages of both. Temporal module is used to model the correlation of sequence samples along the time direction, including the time domain or frequency domain. Time domain analysis often uses networks such as RNN, TCN and Transformer, and frequency domain analysis often uses Fourier transform, wavelet transform, etc.

Existing STGNN methods are divided into discrete or continuous network architectures. When discrete architecture GNN processes graph data, it usually learns the relationship between nodes by iteratively updating the representation vectors of nodes. These models iterate node representations in discrete time steps. , each time step considers the relationship between a node and its neighbor nodes. Continuous Graph Neural Networks use continuous time and differential equations to model the dynamic changes of graph data. The representation of nodes is no longer updated through discrete time steps, but differential equations are used to describe the changes in node representation over time. With the change of time, this allows the node representation to continuously evolve in the time dimension, thereby better capturing the dynamic changes in the graph data. Whether it is a discrete or continuous architecture, existing methods can be further divided into two categories. One type of model models time series and space separately. For example, the time series information is modeled first and then the spatial module is used to extract spatial information. The other type couples space and time series together to model, such as DCRNN and MTGODE. Other authors refer to the very related self as time then space and time and space.

GNN for time series prediction

Time series forecasting aims to predict future time series values based on historical observations. The article summarizes time series prediction based on GNN from three aspects:

To model spatial (i.e., between variables) dependencies, current research usually uses spectral GNNs, spatial GNNs or a combination of the two to model. Early Spectral GNN prediction models mainly used ChebConv to approximate graph convolution and modeled the spatial dependence between time series through Chebyshev polynomials. For example, STGCN combined temporal convolution and ChebConv layers to capture spatial and temporal patterns. Emphasis on local homogeneity based on Spatial GNN. Early methods such as DCRNN and Graph WaveNet incorporate graph diffusion layers into GRU or temporal convolution for modeling time series data. STGCN(1st) and ST-MetaNet model spatial dependence through GCN and GAT to aggregate information of adjacent time series. SLCNN implements ChebConv and local message passing as global and local convolutions to capture spatial relationships at multiple granularities.
Temporal dependency modeling. Time dependence can be modeled from the time domain and frequency domain. Usually convolutional neural networks and attention mechanisms can analyze the time domain and frequency domain. Recurrent neural networks are mainly used for modeling time domain information.
Fusion of spatiotemporal modules for time series prediction can be divided into two fusion methods. One method splices two modules and models the time and space dependencies separately, and the other method merges the two modules together. Common methods are summarized in the table below.

GNN for timing anomaly detection

Time series anomaly detection aims to identify observations that do not conform to the normal generation mechanism of the data. Identifying abnormal samples usually faces two major challenges: First, compared with normal data, it is difficult to collect a large number of abnormal samples. Second, the fluctuation boundaries of abnormal data are difficult to determine, which brings huge challenges to supervised learning. Therefore, unsupervised anomaly detection technology has received widespread attention. This type of method is usually trained based on normal data or a large amount of normal data and a small number of abnormal samples. The sample anomaly score is calculated through a model (which can be any model, including graph neural networks). Used to indicate the degree of abnormality of the sample. The anomaly score can be calculated by evaluating the reconstruction error, prediction error, and relational difference of the sample. Methods based on reconstruction error usually adopt an autoencoder structure and rely on the assumption that the model can correctly model the data generation mechanism in normal samples, so it can accurately reconstruct normal samples. Abnormal samples do not follow this data mechanism, so the model is not sensitive to abnormal samples. Has a large reconstruction error. of uncertainty. During normal periods, the reconstruction error should be low, while during abnormal periods it should be high. The method based on prediction error is similar to the above method. The model has high prediction accuracy and small prediction error for normal samples, but has low prediction accuracy and large prediction error for abnormal samples. Finally, for methods based on relationship differences, it is considered that the variable correlation of normal samples is significantly different from the variable correlation of abnormal samples. Common methods are summarized in the table below.

GNN for temporal classification

GNN-based time series classification methods can make full use of the power of GNN to capture local and global patterns in time series by converting time series data into graph representations. For univariate time series, existing methods can be divided into two categories: Series2Node and Series2Graph. Series2Graph decomposes a univariate time series into multiple subsequences, each subsequence serves as a node, thereby converting the univariate time series into a graph. The edges in the graph describe the dependencies of different subsequences along the time direction. Finally, through Graph Neural Network (GNN) for graph classification. The Series2Node method treats the time series sample as a single node. The edges of the node describe the relationship between different samples, creating a large graph that provides a complete description of the time series dependencies in the entire data set. Multivariate time series classification has the same paradigm as univariate time series classification, except that multivariate time series requires further consideration of modeling the spatial dependence between different series. The following table summarizes the current related work on GNN for time series classification.

GNN for missing data filling

GNN-based data filling methods better characterize the complex spatiotemporal dependencies in time series data. From the perspective of the task, GNN-based time series interpolation can be roughly divided into two types: in-sample interpolation and out-of-sample interpolation. The former involves filling in missing values in given time series data, while the latter predicts missing values in unseen series. From a methodological perspective, GNNs for time series interpolation can be further divided into deterministic interpolation and probabilistic interpolation. Deterministic interpolation provides a single best estimate for missing values, while probabilistic interpolation takes into account the uncertainty in the interpolation process and provides a possible probability distribution of missing values. The following table summarizes most of the related work on GNN-based time series interpolation to date.