Learning Graph Structures With Transformer for Multivariate Time-Series Anomaly Detection in IoT

By Zekai Chen, Dingshuo Chen, Zixuan Yuan, Xiuzhen Cheng, Xiao Zhang. IEEE Internet of Things Journal, 2021

541 citations recorded by the source corpus.

Many real-world Internet of Things (IoT) systems, which include a variety of Internet-connected sensory devices, produce substantial amounts of multivariate time-series data. Meanwhile, vital IoT infrastructures, such as smart power grids and water distribution networks are frequently targeted by cyberattacks, making anomaly detection an important study topic. Modeling such relatedness is, nevertheless, unavoidable for any efficient and effective anomaly detection system, given the intricate topological and nonlinear connections that are originally unknown among sensors. Furthermore, detecting anomalies in multivariate time series is difficult due to their temporal dependency and stochasticity. This article presented GTA, a new framework for multivariate time-series anomaly detection that involves automatically learning a graph structure, graph convolution, and modeling temporal dependency using a transformer-based architecture. The connection learning policy, which is based on the Gumbel-softmax sampling approach to learn bidirected links among sensors directly, is at the heart of the learning graph structure. To describe the anomaly information flow between network nodes, we introduced a new graph convolution called influence propagation convolution. In addition, to tackle the quadratic complexity barrier, we suggested a multibranch attention mechanism to replace the original multihead self-attention method. Extensive experiments on four publicly available anomaly detection benchmarks further demonstrate the superiority of our approach over alternative state of the arts.

Subfields: Artificial Intelligence, Computer Networks and Communications, Signal Processing

View Paper: https://www.semanticscholar.org/paper/95f5870b18d5f894e4f6ec8490d1a39e0963e79e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mingsheng Long
2. Jianmin Wang
3. Junchi Yan
4. Ming Jin
5. Liang Sun
6. Tian Zhou
7. Ziqing Ma
8. Chao Zhang

In Computer Networks and Communications

1. Geoffrey I. Webb
2. Cesare Alippi
3. Jihun Yi
4. Haixu Wu
5. Liang Sun
6. Jiehui Xu
7. Jianmin Wang
8. Kukjin Choi

In Signal Processing

1. Junchi Yan
2. Tian Zhou
3. Liang Sun
4. Haixu Wu
5. Mingsheng Long
6. Qingsong Wen
7. Ziqing Ma
8. Jianmin Wang

Citations in the corpus

• Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines (2021, methodology): “In an unsupervised setting, clustering-based methods are simple yet effective choices for grouping the data and detecting the anomalies.”
• DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection (2023, background): “Most of the supervised and semisupervised methods [14, 18, 51, 53, 87, 90] can not handle the challenge of limited labeled data, especially the anomalies are dynamic and new anomalies never observed before may occur.”
• Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy (2021, methodology): “Especially for time series anomaly detection, GTA proposed by Chen et al. (2021) employs the graph structure to learn the relationship among multiple IoT sensors, as well as the Transformer for temporal modeling and the reconstruction criterion for anomaly detection.”
• Deep Learning for Time Series Anomaly Detection: A Survey (2022, unknown): “GTA [34] uses transformers for sequence modelling and a bidirectional graph to learn relationships among multiple IoT sensors.”
• Transformers in Time Series: A Survey (2022, methodology): “GAN GTA [ Chen et al. , 2021c ] combines Transformer with graph-based learning architecture for multivariate time series anomaly detection.”
• A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection (2023, methodology): “The scorer compares the forecasts against the real observed signals to compute discrepancies such as absolute error [36] or mean-squared error [89].”