Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network

By Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, Dan Pei. Knowledge Discovery and Data Mining, 2019

1,893 citations recorded by the source corpus.

Industry devices (i.e., entities) such as server machines, spacecrafts, engines, etc., are typically monitored with multivariate time series, whose anomaly detection is critical for an entity's service quality management. However, due to the complex temporal dependence and stochasticity of multivariate time series, their anomaly detection remains a big challenge. This paper proposes OmniAnomaly, a stochastic recurrent neural network for multivariate time series anomaly detection that works well robustly for various devices. Its core idea is to capture the normal patterns of multivariate time series by learning their robust representations with key techniques such as stochastic variable connection and planar normalizing flow, reconstruct input data by the representations, and use the reconstruction probabilities to determine anomalies. Moreover, for a detected entity anomaly, OmniAnomaly can provide interpretations based on the reconstruction probabilities of its constituent univariate time series. The evaluation experiments are conducted on two public datasets from aerospace and a new server machine dataset (collected and released by us) from an Internet company. OmniAnomaly achieves an overall F1-Score of 0.86 in three real-world datasets, signicantly outperforming the best performing baseline method by 0.09. The interpretation accuracy for OmniAnomaly is up to 0.89.

Subfields: Artificial Intelligence, Computer Networks and Communications, Signal Processing

View Paper: https://www.semanticscholar.org/paper/440d248d148f7e36dad232e48f1c5c1cbc556d86

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mingsheng Long
2. Jianmin Wang
3. Donghao Luo
4. Xue Wang
5. Tian Zhou
6. Liang Sun
7. Guansong Pang
8. Haixu Wu

In Computer Networks and Communications

1. Donghao Luo
2. Xue Wang
3. Haixu Wu
4. Jianmin Wang
5. Mingsheng Long
6. Liang Sun
7. Tian Zhou
8. Nicholas R. Jennings

In Signal Processing

1. Donghao Luo
2. Xue Wang
3. Qiang Xu
4. Ailing Zeng
5. Mingsheng Long
6. Jianmin Wang
7. Tian Zhou
8. Liang Sun

Citations in the corpus

• Deep Isolation Forest for Anomaly Detection (2022, methodology): “Omni [26] are employed in the experiments on time-series data.”
• Multivariate Time-series Anomaly Detection via Graph Attention Network (2020, methodology): “Following the evaluation strategy in [4], we treat the whole segment as correct if any observation in this segment is detected as anomaly correctly.”
• USAD: UnSupervised Anomaly Detection on Multivariate Time Series (2020, methodology): “On top, the results obtained with SWaT and WADI datasets are presented, whereas the bottom part of the table reports obtained results from the benchmark proposed by [17], using three remaining datasets.”
• Anomaly Detection in Time Series: A Comprehensive Evaluation (2022, methodology): “reconstruction own Tensorflow OmniAnomaly [125] Deep L.”
• MOMENT: A Family of Open Time-series Foundation Models (2024, methodology): “We also note that the UCR anomaly archive was proposed as an improvement over pre-existing anomaly detection datasets such as the SMD (Su et al., 2019), and SMAP (Hundman et al., 2018), many of which are also used in TimesNet.”
• An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series (2021, methodology): “S2: Left: Signals from three channels from the SMD dataset, showing strong correlations across channels and time.”
• Deep Learning for Time Series Anomaly Detection: A Survey (2022, neutral): “2) LSTM-VAE [143] T Un P OmniAnomaly [163] T Un W ✓ Point + Subseq ✓ STORN [159] ST Un P Point GGM-VAE [76] T Un W Subseq SISVAE [114] T Un W Point VAE-GAN [138] T Semi W Point VELC [186] T Un TopoMAD [81] ST Un W Subseq ✓ PAD [34] T Un W Subseq InterFusion [119] ST Un W ✓ Subseq MT-RVAE* [177] ST U”
• Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress (2020, background): “In [18] we show a gallery of dozens of additional examples from Yahoo [5], Numenta [6], NASA [2] and Pei’s Lab (OMNI) [3] that yield to one line solutions.”
• A Review on Outlier/Anomaly Detection in Time Series Data (2020, background): “an outlier. Given that the results directly vary depending on that selection and few techniques give an automatic way to determine the threshold [Hundman et al. 2018;Lu et al. 2018;Siffer et al. 2017;Su et al. 2019], an interesting future line of research would be to deepen on the dynamic and adapti”
• TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis (2022, methodology): “We compare models on five widely-used anomaly detection benchmarks: SMD (Su et al., 2019), MSL (Hundman et al., 2018), SMAP (Hundman et al., 2018), SWaT (Mathur & Tippenhauer, 2016), PSM (Abdulaal et al., 2021), covering service monitoring, space & earth exploration, and water treatment applications.”
• Timeseries Anomaly Detection using Temporal Hierarchical One-Class Network (2020, methodology): “Other more powerful deep generative models, such as the recurrent variational autoencoder [28], and variants of the generative adversarial network (GAN) (e.g., MAD-GAN [13] and BeatGAN [33]) have also been proposed.”
• Learning Graph Structures With Transformer for Multivariate Time-Series Anomaly Detection in IoT (2021, methodology): “M is also reported as the number of features or variables in some literature [6], [13], [4].”
• ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis (2024, methodology): “For anomaly detection, we compare models on five widely-used benchmarks: SMD (Su et al., 2019), SWaT (Mathur & Tippenhauer, 2016), PSM (Abdulaal et al., 2021), MSL and SMAP (Hundman et al., 2018).”
• Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines (2021, methodology): “Meanwhile, OmniAnomaly [68] connects stochastic latent variables in the middle of encoder and decoder with a linear Gaussian state-space model to model the temporal dependencies with inherent stochasticity.”
• TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data (2022, methodology): “The OmniAnomaly, CAE-M and MSCRED models use sequential observations as input, allowing these methods to retain the temporal information.”
• Practical Approach to Asynchronous Multivariate Time Series Anomaly Detection and Localization (2021, methodology): “Nevertheless, the datasets used in [13, 28] ranged from 10 to 50 features.”
• Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding (2021, methodology): “However, previous works [2, 14, 19, 30, 34] mainly model one of the dependencies ( e.g. , intermetric dependency, through latent metric embedding), which could limit their capability on learning normal MTS patterns and detecting anomalies , especially those violating the other kind of dependency (…”