An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series

By Astha Garg, Wenyu Zhang, Jules Samaran, Ramasamy Savitha, Chuan-Sheng Foo. IEEE Transactions on Neural Networks and Learning Systems, 2021

337 citations recorded by the source corpus.

Several techniques for multivariate time series anomaly detection have been proposed recently, but a systematic comparison on a common set of datasets and metrics is lacking. This article presents a systematic and comprehensive evaluation of unsupervised and semisupervised deep-learning-based methods for anomaly detection and diagnosis on multivariate time series data from cyberphysical systems. Unlike previous works, we vary the model and post-processing of model errors, i.e., the scoring functions independently of each other, through a grid of ten models and four scoring functions, comparing these variants to state-of-the-art methods. In time-series anomaly detection, detecting anomalous events is more important than detecting individual anomalous time points. Through experiments, we find that the existing evaluation metrics either do not take events into account or cannot distinguish between a good detector and trivial detectors, such as a random or an all-positive detector. We propose a new metric to overcome these drawbacks, namely, the composite F-score (Fc1), for evaluating time-series anomaly detection. Our study highlights that dynamic scoring functions work much better than static ones for multivariate time series anomaly detection, and the choice of scoring functions often matters more than the choice of the underlying model. We also find that a simple, channel-wise model—the univariate fully connected auto-encoder, with the dynamic Gaussian scoring function emerges as a winning candidate for both anomaly detection and diagnosis, beating state-of-the-art algorithms.

Subfields: Artificial Intelligence, Computer Networks and Communications, Signal Processing

View Paper: https://www.semanticscholar.org/paper/43033640630a3fb4217f9e880ebd82dff1b96785

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Ming Jin
  2. Geoffrey I. Webb
  3. Cesare Alippi
  4. Shirui Pan
  5. Qingsong Wen
  6. S. Kotsiantis
  7. Raymond J. Mooney
  8. Prem Melville

In Computer Networks and Communications

  1. Geoffrey I. Webb
  2. Cesare Alippi
  3. Shirui Pan
  4. Qingsong Wen

In Signal Processing

  1. Geoffrey I. Webb
  2. Cesare Alippi
  3. Shirui Pan
  4. Qingsong Wen
  5. Bart Goethals
  6. Markus Hegland

Citations in the corpus