【GNN+Anomaly Detection】Deep Anomaly Detection on Attributed Networks

Introduction to the paper

Original title : Deep Anomaly Detection on Attributed Networks
Chinese title : Deep Anomaly Detection based on Attributed Networks
Publication conference : SIAM International Conference on Data Mining
Publication year : 2019-05-06
Author : Kaize Ding Jundong Li Rohit Bhanushali Huan Liu
latex citation :

@inproceedings{ding2019deep,
  title={Deep anomaly detection on attributed networks},
  author={Ding, Kaize and Li, Jundong and Bhanushali, Rohit and Liu, Huan},
  booktitle={Proceedings of the 2019 SIAM International Conference on Data Mining},
  pages={594--602},
  year={2019},
  organization={SIAM}
}

Summary

Attributed networks are ubiquitous and an important part of modern information infrastructure, where additional node attributes complement the original network structure in knowledge discovery. In recent years, detecting abnormal nodes on attribute networks has attracted increasing research attention and has been widely used in various high-impact fields such as network security, finance, and healthcare. However, most existing attempts address the problem of shallow learning mechanisms through ego-network or community analysis or through subspace selection. There is no doubt that these models cannot fully solve the computational challenges of attribute networks. For example, they often suffer from network sparsity and data nonlinearity issues, and cannot capture the complex interactions between different information patterns, thus affecting the performance of anomaly detection. In order to solve the above problems, this paper studies the anomaly detection problem of attribute networks by developing a new deep model. In particular, our proposed deep model:

(1) Explicitly model topology and node properties explicitly for node embedding learning with the popular graph convolutional network (GCN);

(2) is customized to solve the anomaly detection problem by leveraging learned embeddings through deep autoencoders to reconstruct the original data.

The synergy between GCN and autoencoders allows us to measure the reconstruction error of nodes from both structural and attribute perspectives to discover anomalies. Extensive experiments on real-world attribute network datasets demonstrate the effectiveness of our proposed algorithm.

Introduction to attribute networks

Since attribute networks have powerful modeling capabilities in unifying different modal information, the detection of abnormal nodes whose modes deviate significantly from most other nodes in attribute networks has become a research hotspot. Generally speaking, the anomalies of nodes in attribute networks not only depend on the interactions between nodes (topology), but can also be measured by the content disharmony of nodes (node ​​attributes).

Problems

1. Network sparsity - On real-world attribute networks, the network structure can be very sparse; therefore, ego network or community analyzes are difficult to perform as they are highly dependent on observed node interactions.

2. Data nonlinearity—the interactions between nodes and node attributes are highly nonlinear, and existing anomaly detectors based on subspace selection mainly use linear mechanisms to model attribute networks.

3. Complex modal interactions - Attribute networks are very difficult to handle due to the confusing combination of two information sources, which requires a unified feature space to capture their complex interactions for anomaly detection.

Paper contribution

1. Dominant, a principled graph convolutional autoencoder, is developed that seamlessly models attribute networks and performs anomaly detection in a joint framework. In particular, this model can analyze the reconstruction error of nodes from both the structure and attribute perspectives to discover anomalies.
2. Our proposed model is evaluated on various attribute networks from different domains. Experimental results show that the framework proposed in this paper has good performance.

The paper’s approach to solving the above problems:

• Network sparsity, complex modal interactions : using GCN. Taking topology and node attributes as input, discriminative node embeddings are learned by stacking multiple layers of linear units and nonlinear activation functions.
• Data nonlinearity : GCN can analyze the reconstruction error of nodes from both the structure and attribute perspectives to find anomalies.
• Anomaly detection : First use the graph convolutional network as the encoder function to compress the input attribute network into a concise low-dimensional embedding representation; then use the corresponding decoder function to reconstruct the topology structure and node attributes. The reconstruction errors of nodes after the encoder and decoder stages are then used to discover abnormal nodes on the attribute network.

1. Proposed model - Dominant

Formulate the anomaly detection task on attribute networks as a ranking problem:

Attribute network anomaly sorting: Given an attribute network G, the adjacency matrix is ​​represented by A, and the attribute information matrix is ​​represented by position.

The basic building block of Dominant is the deep autoencoder [11], which consists of three basic components

1. Attributed Network Encoder – which seamlessly models network structure and node attributes in a joint framework with GCN’s node embedding representation learning
2. Structural reconstruction decoder - aims to reconstruct the original network topology using learned node embeddings
3. Attribute reconstruction decoder - attempts to reconstruct the observed node attributes using the obtained node embeddings. Then, the node's reconstruction error is utilized to mark anomalies on the attribute network.

Deep autoencoder:

As shown in [32, 37, 17], the difference between original data and estimated data (i.e., reconstruction error) is a powerful indicator of anomalies in instances in a dataset. Specifically, data instances with large reconstruction errors are more likely to be considered anomalies because their patterns deviate significantly from the majority of the data and cannot be accurately reconstructed from the observed data. Among various reconstruction-based anomaly detection methods, deep autoencoders achieve state-of-the-art performance.

Given an input data set The learning process can be described as minimizing the cost function as follows:

m i n { E [ d i s t ( X , D e c ( E n c ( X ) ) ) ] } min\{ E[dist(X,Dec(Enc(X)))] \} min{ E[dist(X,Dec ( E n c ( X )))]}

1. attribute network encoder

   GCN, obtain a new node embedding matrix Z with the shape of（node_size, embedding_size）

2. structure reconstruction decoder

   During reconstruction, the method to determine whether there is a link between node i and node j in the original graph:
   $$p({\hat{A}}_{i,j}=1|z_i,z_j)=sigmoid(z_i,z_j^T)$$
   Principle: To determine the similarity of two vectors in space, that is, just perform the inner product operation on the two vectors. If they are in the same direction, the value is the largest. Then use the sigmoid function to control the dimension between [0,1]. The value obtained in this way can also be used directly as a probability.
   Applied to the matrix, it is:
   $$\hat{A}=sigmoid(Z{Z}^T)$$

   The structural anomalies on the network are then determined through the structural reconstruction error :
   $$R_S=A-\hat{A}$$

3. Attribute reconstruction decoder

   The attribute reconstruction decoder utilizes another graph convolution layer to predict the original node attributes, and finally outputs the reconstructed node embedding $\hat{X}$ , and then through the calculated reconstruction error, we can find anomalies on the attribute network from the attribute perspective:
   $$R_A=X-\hat{X}$$

• abnormal detection

  In order to jointly learn the reconstruction error, the objective function of our proposed depth map convolutional autoencoder can be expressed as:
  $$L=(1-a)R_S+\alpha R_A=(1-a)||A-\hat{A}|{|}_{F^{\prime }}^2+\alpha ||X-\hat{X}|{|}_{F^{\prime }}^2$$
  In this way, the anomaly score of each node can be expressed as:
  $$score(v_i)=(1-a)||a-\widehat{a_i}|{|}_2+\alpha ||x_i-\widehat{x_i}|{|}_2$$

  Specifically, instances with higher scores are more likely to be considered an anomaly; thus we can calculate the anomaly rank based on the corresponding anomaly score.

2. Experiment

• data set:

  • BlogCatalog: BlogCatalog is a blog sharing website. Bloggers in blogcatalog can follow each other and form a social network. Users are associated with a set of tags to describe themselves and their blogs, which are treated as node attributes.
  • Flickr: Flickr is an image hosting and sharing website. Similar to BlogCatalog, users can follow each other and form a social network. A user's node attributes are defined by their assigned labels that reflect their interests.
  • ACM: ACM is another affiliation network from the academic world. It is a citation network. Each paper is regarded as a node on the network, and the links are the citation relationships between different papers. The attributes of each paper are generated from the paper abstract.

• Baseline model:

  • LOF [4] detects anomalies at the context level and only considers node attributes.
  • SCAN [34] is a structure-based detection method that can detect anomalies at the structural level.
  • AMEN [24] uses both attribute and network structure information to detect anomalous neighborhoods. Specifically, the abnormal situation of each node is analyzed from the perspective of the ego network.
  • Radar [17] is a state-of-the-art unsupervised anomaly detection framework for attribute networks. It detects abnormal behavior by describing the residuals of attribute information and its consistency with network information.
  • ANOMALOUS [23] performs joint anomaly detection and attribute selection based on CUR decomposition and residual analysis to detect anomalies on the attribute network.

• parameter:

  • Optimization algorithm: Adam
  • epoch：300
  • learning_rate：0.005
  • Three-layer convolutional layer embedding_size: 64, 32, 16