SSD: A Unified Framework for Self-Supervised Outlier Detection

By Vikash Sehwag, Mung Chiang, Prateek Mittal. 2021

433 citations recorded by the source corpus.

We ask the following question: what training information is required to design an effective outlier/out-of-distribution (OOD) detector, i.e., detecting samples that lie far away from the training distribution? Since unlabeled data is easily accessible for many applications, the most compelling approach is to develop detectors based on only unlabeled in-distribution data. However, we observe that most existing detectors based on unlabeled data perform poorly, often equivalent to a random prediction. In contrast, existing state-of-the-art OOD detectors achieve impressive performance but require access to fine-grained data labels for supervised training. We propose SSD, an outlier detector based on only unlabeled in-distribution data. We use self-supervised representation learning followed by a Mahalanobis distance based detection in the feature space. We demonstrate that SSD outperforms most existing detectors based on unlabeled data by a large margin. Additionally, SSD even achieves performance on par, and sometimes even better, with supervised training based detectors. Finally, we expand our detection framework with two key extensions. First, we formulate few-shot OOD detection, in which the detector has access to only one to five samples from each class of the targeted OOD dataset. Second, we extend our framework to incorporate training data labels, if available. We find that our novel detection framework based on SSD displays enhanced performance with these extensions, and achieves state-of-the-art performance. Our code is publicly available at https://github.com/inspire-group/SSD.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Li Shen
2. Yiyou Sun
3. Mingchen Gao
4. Zhenyi Wang
5. Qiuling Suo
6. Bin Liu
7. Le Fang
8. Tiehang Duan

Citations in the corpus

• Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, methodology): “We adopt a method similar to the state-of-the-art few-shot OOD detection algorithm, named cluster-conditioned detection [52].”
• Out-of-distribution Detection with Deep Nearest Neighbors (2022, methodology): “Other works (Winkens et al., 2020; Sehwag et al., 2021) verify the effectiveness of applying the off-the-shelf multi-view contrastive losses such as SimCLR (Chen et al., 2020) and SupCon (Khosla et al., 2020) for OOD detection.”
• CMG: A Class-Mixed Generation Approach to Out-of-Distribution Detection (2022, background): “Some researchers also tried to improve contrastive learning based methods [49] and proposed distance-based methods [34].”
• Mitigating Neural Network Overconfidence with Logit Normalization (2022, methodology): “Contrastive learning meth-ods are also employed for the OOD detection task (Tack et al., 2020; Sehwag et al., 2021; Ming et al., 2022b), which can be computationally more expensive to train than ours.”
• ADBench: Anomaly Detection Benchmark (2022, background): “We may consider self-supervision [140, 158, 161, 179] and transfer learning [33] to improve tabular AD as well.”