【读论文】Graph Neural Network-Based Anomaly Detection in Multivariate Time Series

1. The main research content of this article

Graph Neural Network-Based Anomaly Detection in Multivariate Time Series: Multivariate Time Series Anomaly Detection Based on Graph Neural Network

Goal : To develop specific methods for multivariate time-series data that explicitly capture graphs of relationships between sensors.

Weaknesses of Existing Approaches : Existing approaches do not explicitly learn the structure of existing relationships between variables, or use them to predict the expected behavior of time series.

Improvements for this defect : In order to take full advantage of the complex relationship between sensors in multivariate time series, graph neural networks (GNNs) are used to learn the relationship graph between sensors.

Graph-based methods provide a way to model the relationship between sensors by representing interdependencies with edges.

• In general, GNN assumes that the state of a node is affected by the state of its neighbors;
• Graph Convolution Networks (GCNs) model the feature representation of nodes by aggregating representations of their one-step neighbors;
• Graph attention networks (GATs) use attention functions to compute different weights for different neighbors during aggregation.

Proposed method :Graph Deviation Network (GDN), the method learns a graph of relationships between sensors and detects biases from these patterns. The method consists of four main parts:

1. Sensor Embedding, sensor embedding : use the embedding vector to flexibly capture the unique characteristics of each sensor;
2. Graph Structure Learning, graph structure learning : graph structure learning learns the relationship between sensor pairs and encodes them as edges in the graph;
3. Graph Attention-Based Forecasting, prediction based on graph attention : predict the future behavior of sensors based on the attention function of adjacent sensors in the graph;
4. Graph Deviation Scoring, Graph Deviation Scoring : Identify and account for deviations in sensor relationships learned from graphs, and localize and account for these deviations.

2. Graph Deviation Network (GDN)

1. Problem Statement

Training data : $N$ sensors at$T_{train}$Data during this period: $s_{train}=[s_{train}^{(1)},s_{train}^{(2)},...,s_{train}^{(T_{train})}]$ ，其中，$s_{train}^{(t)}$is a $N-$ dimensional vector, denoted in$t$ time$The\; values\; of\; N$ sensors.
Following the usual unsupervised anomaly detection formulation, it is assumed that the training data contains only normal data.

Our goal is to be able to detect anomalies in the test data represented as: $s_{train}=[s_{test}^{(1)},s_{test}^{(2)},...,s_{test}^{(T_{test})}]$

The output of the algorithm is a set of size $T_{test}$The binary labels of , this set of labels indicates that at each time $t$ whether an exception occurred. a ( t ) ∈ { 0 , 1 } a(t)\in\{0,1\}a(t)∈{ 0,1 } , where$a\left(t\right)=1$ meansttAn exception occurred at time$t\; .$

2. Overview

The method consists of four main parts:

1. Sensor Embedding, sensor embedding : use the embedding vector to flexibly capture the unique characteristics of each sensor;
2. Graph Structure Learning, graph structure learning : graph structure learning learns the relationship between sensor pairs and encodes them as edges in the graph;
3. Graph Attention-Based Forecasting, prediction based on graph attention : predict the future behavior of sensors based on the attention function of adjacent sensors in the graph;
4. Graph Deviation Scoring, Graph Deviation Scoring : Identify and account for deviations in sensor relationships learned from graphs, and localize and account for these deviations.

(1) Sensor Embedding, sensor embedding

Introduce an embedding vector for each sensor to represent its features: $v_i\in R^d$ , for i ∈ { 1 , 2 , . . . , N } i \in \{1,2,...,N\} i∈{ 1,2,...,N} 。

These embedding vectors $v_i$The similarity between indicates the similarity of behavior. Therefore, there should be a high correlation between sensors with similar embedded values.

(2) Graph Structure Learning, graph structure learning

Graph structure learning will learn a weighted directed graph whose nodes represent sensors and edges represent dependencies between sensors.
For sensor $For\; i$ , we calculate sensor$The\; embedding\; vector\; of\; i$ and its candidate relationship$C_i$The similarity (normalized dot product) of $e_{ji}$ ：
$$e_{ji}=\frac{{v_i{v}_i}^T{v}_j{v}_j}{||v_i{v}_i||\cdot ||v_j{v}_j||}forj\in C_i$$

Then select the former $k$ such normalized dot products,$The\; value\; of\; k$ can be chosen by the user according to the desired degree of sparsity.
A ji = 1 { j ∈ T op K ( eki : k ∈ C i ) } A_{ji} = 1\{j\in TopK({e_{ki}:k\in C_i})\}Aji​=1{ j∈Top of K ( e _to:k∈Ci​)}
where,1{·}is an indicative function, namely1{值为真的表达式} = 1,1{值为假的表达式} = 0.

In the absence of prior information, sensor $Candidate\; relations\; for\; i$ are all sensors except itself.

(3) Graph Attention-Based Forecasting, prediction based on graph attention

at time $t$ , we based on historical time series data with size$The\; sliding\; window\; of\; w$ , the input of the model is defined as${xx}^{(t)}:=[s^{(t-w)}{s}^{(t-w)},s^{(t-w+1)}{s}^{(t-w+1)},...,s^{(t-1)}{s}^{(t-1)}]$. The target output that the model needs to predict is the sensor data at the current moment, that is,$s^{(t)}$ 。

To capture different behaviors of sensors, we introduce a graph attention-based feature extractor to fuse node information with neighboring nodes based on the learned graph structure:
$${zz}_i^{(t)}=ReLU(a_{i,i}WW{xx}_i^{\left(t\right)}+\sum _{j\in N(i)}{a}_{i,j}WW{xx}_j^{\left(t\right)})$$ , where${xx}_i^{(t)}$For model input, N ( i ) = { j ∣ A ji > 0 } N(i) = \{ j | A_{ji} > 0 \}N(i)={ j∣Aji​>0} ，$W$ is the weight matrix obtained from training, pay attention to the coefficient$a_{i,j}$The calculation formula is:

In this way, we get the representation of all N nodes, namely { z 1 ( t ) , z 2 ( t ) , .{ zz1(t)​,zz2(t)​,...,zzN(t)​}

For each ${zz}_i^{\left(t\right)}$, we associate it with the corresponding embedding vector ${vv}_i$Perform element-wise multiplication (denoted as $\circ$ ), and take the calculation results of all nodes as the input of the N-dimensional fully connected layer to predictttVector s ( t ) \pmb{s}^{(t)}of sensor values ​​at time$t$${ss}^{(t)}$ ：
$${\hat{s}}^{(t)}{\hat{s}}^{\left(t\right)}=f_{}([{vv}_1\circ {zz}_1^t,{vv}_2\circ {zz}_2^t,...,{vv}_N\circ {zz}_N^t])$$
We hope that the predicted output of the model is as close as possible to the real value, so use the predicted output${\hat{s}}^{\left(t\right)}{\hat{s}}^{\left(t\right)}$ and observed data$s^{\left(t\right)}{s}^{\left(t\right)}$ is minimized as a loss function:
$$L_{MSE}=\frac{1}{T_{train-w}}\sum _{t=w+1}^{T_{train}}||{\hat{s}}^{(t)}{\hat{s}}^{\left(t\right)}-s^{\left(t\right)}{s}^{(t)}|{|}_2^2$$

(4) Graph Deviation Scoring, graph deviation score

Given the learned relationships, we wish to detect and explain anomalies that deviate from these relationships.

sensor $i$ inttDeviation between predicted behavior and observed behavior at time$t$$Er{r}_i(t)=|s_i^{(t)}-{\hat{s}}^{(t)}|$

The deviation of different sensors may have different scales, so the deviation of each sensor is normalized: $a_i\left(t\right)=\frac{Er{r}_i(t)-{\tilde{m}}_i}{{\tilde{p}}_i}$\widetilde{\mu} _i${\tilde{m}}_i$ 是$Er{r}_i\left(t\right)$ ,${\tilde{p}}_i$ 是$Er{r}_i\left(t\right)$ Inter-quartile range (IQR).

Inter-Quartile Range, IQR is the difference between the 1/4th and 3/4th of a distribution or set of values, ie IQR = Q3 - Q1, and is a strong measure of the spread of the distribution.

To calculate $The\; overall\; abnormal\; situation\; at\; time\; t$ , we use$The\; max$$function$ aggregates sensors (since anomalies will only affect a small subset of sensors, or even a single sensor):
$$A(t)=\underset{i}{\max }{a}_i\left(t\right)$$
if$A\left(t\right)$ exceeds the set threshold, andttThe data at time$t\; is\; marked\; as\; abnormal.$