Further Analysis of Outlier Detection with Deep Generative Models

By Ziyu Wang, Bin Dai, David Paul Wipf, Jun Zhu. 2020

45 citations recorded by the source corpus.

The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling. In this work, we present a possible explanation for this phenomenon, starting from the observation that a model's typical set and high-density region may not conincide. From this vantage point we propose a novel outlier test, the empirical success of which suggests that the failure of existing likelihood-based outlier tests does not necessarily imply that the corresponding generative model is uncalibrated. We also conduct additional experiments to help disentangle the impact of low-level texture versus high-level semantics in differentiating outliers. In aggregate, these results suggest that modifications to the standard evaluation practices and benchmarks commonly applied in the literature are needed.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Minlie Huang
3. Maosong Sun
4. Xipeng Qiu
5. Ji-rong Wen
6. Yixuan Li
7. Wayne Xin Zhao
8. Jie Tang

In Computer Vision and Pattern Recognition

1. Yixuan Li
2. Yiyou Sun
3. Chuan Guo
4. Andrew Geng
5. Rui Huang
6. Zhiyuan Liu
7. Laurent Charlin
8. Oleksiy Ostapenko

Citations in the corpus

• Continual Learning via Local Module Composition (2021, background): “Several recent works have shown that deep generative models often mistakenly assign high likelihood values to outlier points [68, 36, 95].”
• Pre-Trained Models: Past, Present and Future (2021, methodology): “Another kind of works (Wang et al., 2020b) can use adapters trained on different knowledge sources with extra annotations to distinguish where the knowledge is from.”
• On the Importance of Gradients for Detecting Distributional Shifts in the Wild (2021, background): “A plethora of literature has emerged to utilize generative models for OOD detection [19, 37, 39, 40, 45, 46, 48].”
• ADBench: Anomaly Detection Benchmark (2022, background): “AD algorithms aim to identify data instances that deviate significantly from the majority of data objects [59, 139, 146, 160], and numerous methods have been developed in the last few decades [3, 85, 102, 103, 129, 156, 172, 198].”
• ReAct: Out-of-distribution Detection With Rectified Activations (2021, background): “A plethora of literature has emerged to utilize generative models for OOD detection [24, 40, 45, 46, 52, 56].”