Why Normalizing Flows Fail to Detect Out-of-Distribution Data

By Polina Kirichenko, Pavel Izmailov, Andrew Gordon Wilson. 2020

362 citations recorded by the source corpus.

Detecting out-of-distribution (OOD) data is crucial for robust machine learning systems. Normalizing flows are flexible deep generative models that often surprisingly fail to distinguish between in- and out-of-distribution data: a flow trained on pictures of clothing assigns higher likelihood to handwritten digits. We investigate why normalizing flows perform poorly for OOD detection. We demonstrate that flows learn local pixel correlations and generic image-to-latent-space transformations which are not specific to the target image dataset. We show that by modifying the architecture of flow coupling layers we can bias the flow towards learning the semantic structure of the target data, improving OOD detection. Our investigation reveals that properties that enable flows to generate high-fidelity images can have a detrimental effect on OOD detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

ReAct: Out-of-distribution Detection With Rectified Activations (2021, background): “A plethora of literature has emerged to utilize generative models for OOD detection [24, 40, 45, 46, 52, 56].”
CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows (2021, background): “Recent works [34, 15] show that these models tend to capture only low-level correlations instead of relevant semantic information.”
On the Importance of Gradients for Detecting Distributional Shifts in the Wild (2021, background): “A plethora of literature has emerged to utilize generative models for OOD detection [19, 37, 39, 40, 45, 46, 48].”
CutPaste: Self-Supervised Learning for Anomaly Detection and Localization (2021, background): “Anomaly detection under one-class classification setting, where we assume only the normal data is given during the training, has been widely studied [49, 56, 46, 63, 66, 13, 42, 36, 27].”
A Unifying Review of Deep and Shallow Anomaly Detection (2020, background): “Recent work suggests that one reason for this seems to be that the likelihood in current flow models is dominated by low-level features due to specific network architecture inductive biases [243], [290].”
Low-Light Image Enhancement with Normalizing Flow (2021, background): “However, since the classical normalizing ﬂow is biased towards learning image graphical properties such as local pixel correlations (Kirichenko, Izmailov, and Wilson 2020), it may fail to model some global image properties like the color saturation, which can undermine the performance when applying…”