Generative Ensembles for Robust Anomaly Detection

By Hyun-Jae Choi, Eric Jang. 2018

85 citations recorded by the source corpus.

Deep generative models are capable of learning probability distributions over large, high-dimensional datasets such as images, video and natural language. Generative models trained on samples from $p(x)$ ought to assign low likelihoods to out-of-distribution (OoD) samples from $q(x)$, making them suitable for anomaly detection applications. We show that in practice, likelihood models are themselves susceptible to OoD errors, and even assign large likelihoods to images from other natural datasets. To mitigate these issues, we propose Generative Ensembles, a model-independent technique for OoD detection that combines density-based anomaly detection with uncertainty estimation. Our method outperforms ODIN and VIB baselines on image datasets, and achieves comparable performance to a classification model on the Kaggle Credit Fraud dataset.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (2018, neutral): “Choi and Jang [10] and Shalev et al. [44] employ model ensembling to further improve detection reliability.”
Do Deep Generative Models Know What They Don't Know? (2018, background): “The results we present in Equation 6 do not apply to Choi & Jang (2018)’s models since they use afﬁne coupling layers in their Glow, making it NVP.”
Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting (2019, methodology): “The ability to characterize the bias of a generative model is an important step towards using these models to guide decisions in high-stakes applications under uncertainty [53, 54], such as healthcare [55–57] and anomaly detection [58, 59].”
Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection (2019, methodology): “…as well as three state-of-the-art meth-ods for unsupervised OoD detection (all described in Section 4): (1) The generative ensemble based method by Choi & Jang (2018) composed of ﬁve independently trained models ( WAIC ), (2) the likelihood ratio method by Ren et al. (2019) ( LLR ), using…”
Learning Energy-Based Models in High-Dimensional Spaces with Multiscale Denoising-Score Matching (2019, background): “Choi and Jang (2018) and Nalisnick et al.”
Annealed Denoising Score Matching: Learning Energy-Based Models in High-Dimensional Spaces (2019, background): “, 2010), outlier/novelty detection (Zhai et al., 2016; Choi and Jang, 2018), invariant pattern recognition, Bayesian reasoning (Welling and Teh, 2011) which relies on good data priors, and many others.”
Unsupervised Class-Incremental Learning Through Confusion (2021, methodology): “To address this problem, research related to out-of-distribution (OOD) detection have utilized supervised methods (Liang et al. (2017); Alemi et al. (2018)) and unsupervised methods (Choi & Jang (2018); Hendrycks et al. (2018); Serrà et al. (2019)).”
MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space (2021, background): “Several strategies have been proposed to mitigate this issue, such as improved metrics [5], likelihood ratio [39, 42], and modified training techniques [17].”
Deep Learning for Anomaly Detection: A Survey (2019, background): “In recent times models with good interpret-ability are proposed and shown to produce state-of-the-art performance (Gugulothu et al., Amarasinghe et al. [2018b], Choi [2018]).”
Hybrid Models with Deep and Invertible Features (2019, background): “Firstly, recent work has shown that deep generative models can assign higher likelihood to OOD inputs (Nalisnick et al., 2019; Choi & Jang, 2018), meaning that our rejection rule is not guaranteed to work in all settings.”
Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data (2020, background): “The selection is inspired by the ﬁnding in the line of works that uses a generative model for OoD detection [32, 26, 4].”