CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances

By Jihoon Tack, Sangwoo Mo, Jongheon Jeong, Jinwoo Shin. 2020

745 citations recorded by the source corpus.

Novelty detection, i.e., identifying whether a given sample is drawn from outside the training distribution, is essential for reliable machine learning. To this end, there have been many attempts at learning a representation well-suited for novelty detection and designing a score based on such representation. In this paper, we propose a simple, yet effective method named contrasting shifted instances (CSI), inspired by the recent success on contrastive learning of visual representations. Specifically, in addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Based on this, we propose a new detection score that is specific to the proposed training scheme. Our experiments demonstrate the superiority of our method under various novelty detection scenarios, including unlabeled one-class, unlabeled multi-class and labeled multi-class settings, with various image benchmark datasets.

Subfields: Artificial Intelligence, Control and Systems Engineering, Epidemiology, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Epidemiology

In Representation & Self-Supervised Learning

Citations in the corpus

The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (2022, methodology): “Spurred by this challenge, we propose an anomaly detection task (Hendrycks & Gimpel, 2017; Tack et al., 2020): Can we detect when the true reward starts to drop, while maintaining a low false positive rate in benign cases?”
Unsolved Problems in ML Safety (2021, background): “[178] Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin.”
A Theoretical Study on Solving Continual Learning (2022, methodology): “A strong OOD detection method CSI in [6] helps produce very strong CIL results.”
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection (2022, background): “5 2,780 ☼ CSI [68] (NeurIPS’20) N/A N/A N/A N/A N/A N/A 75.”
CutPaste: Self-Supervised Learning for Anomaly Detection and Localization (2021, methodology): “Following [54], we adopt a two-stage framework for building an anomaly detector, where in the first stage we learn deep representations from normal data and then construct an one-class classifier using learned representations.”
Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP (2021, positive): “CAC was chosen as it is compatible with CLIP and is on average the second best performing baseline after CSI.”
CMG: A Class-Mixed Generation Approach to Out-of-Distribution Detection (2022, methodology): “This loss cannot be used by other OOD methods since they have no OOD data available [3, 21, 53].”
Out-of-distribution Detection with Deep Nearest Neighbors (2022, methodology): “These methods include CSI (Tack et al., 2020) and SSD+ (Sehwag et al.”
Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “3: Self-Supervised Learning [142], [143], [144], [145], [146]”
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning (2023, methodology): “For all the experiments of the proposed methods, we use the values chosen by the original CSI [58].”
VOS: Learning What You Don't Know by Virtual Outlier Synthesis (2022, methodology): “…in MSP (Hendrycks & Gimpel, 2017), ODIN (Liang et al., 2018), Generalized ODIN (Hsu et al., 2020), Mahalanobis distance (Lee et al., 2018b), CSI (Tack et al., 2020), energy score (Liu et al., 2020a) and gram matrices (Sastry & Oore, 2020) and apply them accordingly on the classiﬁcation branch…”
Mitigating Neural Network Overconfidence with Logit Normalization (2022, methodology): “Contrastive learning meth-ods are also employed for the OOD detection task (Tack et al., 2020; Sehwag et al., 2021; Ming et al., 2022b), which can be computationally more expensive to train than ours.”
A Unifying Review of Deep and Shallow Anomaly Detection (2020, methodology): “The major approaches to deep AD include deep AE variants [44], [51], [54], [125]–[135], deep one-class classification [136]–[145], methods based on DGMs, such as GANs [50], [56], [146]–[151], and recent self-supervised methods [152]–[156].”
Online Continual Learning through Mutual Information Maximization (2022, background): “Like in (Tack et al., 2020), we also give pseudo classes to pseudo samples created by different rotations, which produce better features.”
Continual Learning Based on OOD Detection and Task Masking (2022, methodology): “0 and then decay it by cosine scheduler [44] after 10 epochs without restart as in [11, 65].”