A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data

By Markus Goldstein, Seiichi Uchida. 2016

983 citations recorded by the source corpus.

Anomaly detection is the process of identifying unexpected items or events in datasets, which differ from the norm. In contrast to standard classification tasks, anomaly detection is often applied on unlabeled data, taking only the internal structure of the dataset into account. This challenge is known as unsupervised anomaly detection and is addressed in many practical applications, for example in network intrusion detection, fraud detection as well as in the life science and medical domain. Dozens of algorithms have been proposed in this area, but unfortunately the research community still lacks a comparative universal evaluation as well as common publicly available datasets. These shortcomings are addressed in this study, where 19 different unsupervised anomaly detection algorithms are evaluated on 10 different datasets from multiple application domains. By publishing the source code and the datasets, this paper aims to be a new well-funded basis for unsupervised anomaly detection research. Additionally, this evaluation reveals the strengths and weaknesses of the different approaches for the first time. Besides the anomaly detection performance, computational effort, the impact of parameter settings as well as the global/local anomaly detection behavior is outlined. As a conclusion, we give an advise on algorithm selection for typical real-world tasks.

Subfields: Artificial Intelligence, Computer Networks and Communications, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Signal Processing

Citations in the corpus

SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, background): “, [19], [10]) which could enhance our current framework.”
Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors (2018, background): “In the biggest comparison of outlier detection methods [25] the proximity-based approaches are demonstrated to be the most effective strategy.”
CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows (2021, background): “power-line or acoustic) can be accomplished using a common k-nearest-neighbor or more advanced clustering methods [10], it is less trivial for high-resolution images.”
Progress in Outlier Detection Techniques: A Survey (2019, background): “[97], compared COF and LOF, and it was found that the LOF spherical density estimation was a poor choice for efficiently detecting outliers.”
Artificial Intelligence in Pharmaceutical Technology and Drug Delivery Design (2023, background): “Unsupervised anomaly detection techniques, such as the local outlier factor (LOF) or isolation forest, can help highlight abnormal patterns or data points that warrant further investigation [48].”
Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study (2019, background): “Although unsupervised techniques are more ﬂexible than the others [40], most of them rely on the assumption that normal instances are far more frequent than abnormal ones [34].”
Use of Machine-Learning Approaches to Predict Clinical Deterioration in Critically Ill Patients: A Systematic Review (2017, background): “On the other hand, other newer monitoring technologies based on statistical machine-learning theory are available, which provide early recognition of patient health deterioration [8,9].”
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding (2018, background): “alizability, or computational expense [9, 16] (see [9] for a survey of anomaly detection approaches).”
Deep Learning for Anomaly Detection: A Survey (2019, background): “The hidden layers of deep neural network aim to capture these intrinsic properties within the dataset (Goldstein and Uchida [2016]).”
HRN: A Holistic Approach to One Class Learning (2020, background): “They almost exclusively use the neural network representation learning capability to generate a latent representation of the given class [1, 14, 16, 25, 51, 65, 68, 71, 83, 92, 95, 69, 73, 26, 82].”
A comprehensive survey of anomaly detection techniques for high dimensional big data (2020, background): “Anomaly detection is an important technique for recognizing fraud activities, suspicious activities, network intrusion, and other abnormal events that may have great significance but are difficult to detect [25].”
ADBench: Anomaly Detection Benchmark (2022, background): “Although there are already some benchmark and evaluation works for tabular AD [25, 38, 42, 53, 166], they generally have the limitations as follows: ( i ) primary emphasis on unsupervised methods only without including emerging (semi-)supervised AD methods; ( ii ) limited analysis of the algorithm…”
SentiNet: Detecting Physical Attacks Against Deep Learning Systems (2018, background): “Works have shown techniques to detect anomalous high dimensional data using one class neural networks (see, e.g., [7], [16]) which could enhance our current framework.”
Anomaly Detection for IoT Time-Series Data: A Survey (2020, background): “There are two primary ways in which anomalous data may be reported [6], [48].”