Exploring the Limits of Out-of-Distribution Detection

By Stanislav Fort, Jie Jessie Ren, Balaji Lakshminarayanan. 2021

428 citations recorded by the source corpus.

Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks.

Subfields: Artificial Intelligence, Anomaly & Outlier Detection

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Quoc V. Le
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Jianfeng Gao
  5. Percy Liang
  6. Sergey Levine
  7. Trevor Darrell
  8. Neil Houlsby

In Anomaly & Outlier Detection

  1. Stanislav Fort
  2. Yixuan Li
  3. Zhen Fang
  4. Sepideh Esmaeilpour
  5. Dawn Song
  6. Jie Lu
  7. Lei Shu
  8. Mantas Mazeika

Citations in the corpus