A New Local Distance-Based Outlier Detection Approach for Scattered Real-World Data

By Ke Zhang, Marcus Hutter, Huidong Jin. Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2009

425 citations recorded by the source corpus.

Detecting outliers which are grossly different from or inconsistent with the remaining dataset is a major challenge in real-world KDD applications. Existing outlier detection methods are ineffective on scattered real-world datasets due to implicit data patterns and parameter setting issues. We define a novel Local Distance-based Outlier Factor (LDOF) to measure the outlier-ness of objects in scattered datasets which addresses these issues. LDOF uses the relative location of an object to its neighbours to determine the degree to which the object deviates from its neighbourhood. We present theoretical bounds on LDOF's false-detection probability. Experimentally, LDOF compares favorably to classical KNN and LOF based outlier detection. In particular it is less sensitive to parameter values.

Subfields: Artificial Intelligence, Civil and Structural Engineering, Epidemiology

View Paper: https://www.semanticscholar.org/paper/aa7128ed81808cad1ce19fa122a30a616eca2914

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Arthur Zimek
2. Jörg Sander
3. Ricardo José Gabrielli Barreto Campello
4. Erich Schubert
5. Peer Kröger
6. Hans-Peter Kriegel
7. Davoud Moulavi
8. Hongzhi Wang

In Civil and Structural Engineering

1. Jörg Sander
2. Mohamed Hammad
3. Hans-Peter Kriegel
4. Venkatesh Saligrama
5. Ricardo José Gabrielli Barreto Campello
6. Chunhua Shen
7. Peer Kröger
8. Manqi Zhao

In Epidemiology

1. Arthur Zimek
2. Jörg Sander
3. Ricardo José Gabrielli Barreto Campello
4. Mohamed Hammad
5. Mohamed Jaward Bah
6. Hongzhi Wang
7. Davoud Moulavi
8. Hans-Peter Kriegel

Citations in the corpus

• Deep Learning for Anomaly Detection (2020, background): “2x. 16 Pang, et al. effective distance-based anomaly measures introduced,e.g., DB outliers [73, 74], k-nearest neighbor distance [126, 127], average k-nearest neighbor distance [9], relative distance [174], and random nearest neighbor distance [117, 145]. One major limitation of these traditional di”
• A survey on unsupervised outlier detection in high‐dimensional numerical data (2012, methodology): “27,28,71,72,75,78,138,148 and LOCI including the approximate aLOCI [102]—also some spatial outlier detection methods such as those in refs.”
• Progress in Outlier Detection Techniques: A Survey (2019, methodology): “[76] proposed a local distance-based outlier detection method called the Local Distance-based Outlier Factor (LDOF).”
• LoOP: local outlier probabilities (2009, methodology): “ABOD is also quite stable, but at a lower level than LoOP or LDOF.”
• Anomaly Detection with Score functions based on Nearest Neighbor Graphs (2009, methodology): “(K-NN) anomaly detection approach is presented in [3, 8].”
• Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection (2015, methodology): “The local distance-based outlier detection (LDOF) approach [Zhang et al. 2009] merges the notion of local outlierness with the distance-based notion of outliers.”