Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data

By Yen-Chang Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira. Computer Vision and Pattern Recognition, 2020

700 citations recorded by the source corpus.

Deep neural networks have attained remarkable performance when applied to data that comes from the same distribution as that of the training set, but can significantly degrade otherwise. Therefore, detecting whether an example is out-of-distribution (OoD) is crucial to enable a system that can reject such samples or alert users. Recent works have made significant progress on OoD benchmarks consisting of small image datasets. However, many recent methods based on neural networks rely on training or tuning with both in-distribution and out-of-distribution data. The latter is generally hard to define a-priori, and its selection can easily bias the learning. We base our work on a popular method ODIN, proposing two strategies for freeing it from the needs of tuning with OoD data, while improving its OoD detection performance. We specifically propose to decompose confidence scoring as well as a modified input pre-processing method. We show that both of these significantly help in detection performance. Our further analysis on a larger scale image dataset shows that the two types of distribution shifts, specifically semantic shift and non-semantic shift, present a significant difference in the difficulty of the problem, providing an analysis of when ODIN-like strategies do or do not work.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ae9bf201f128cabaa4350b54ff6607525c736cd5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Out-of-distribution Detection with Deep Nearest Neighbors (2022, methodology): “GODIN (Hsu et al., 2020) is trained using a DeConf-C loss, which does not involve contrastive loss either.”
Increasing the Cost of Model Extraction with Calibrated Proof of Work (2022, methodology): “The defenses introduced by Kariyappa et al. assume knowledge of attackers’ OOD data that are generally hard to deﬁne a-priori and the selection of such data can easily bias learning (Hsu et al., 2020).”
CODEs: Chamfer Out-of-Distribution Examples against Overconfidence Issue (2021, methodology): “Recent works improve OOD detection by using the ODIN score [31, 23], Mahalanobis distance [28], energy score [32], ensemble of multiple classifiers [42, 48], residual flow [51], generative models [38], self-supervised learning [20, 35] and gram matrices [39].”
Adversarial Reciprocal Points Learning for Open Set Recognition (2021, background): “With the definition given in [49], real-B has a semantic shift from real-A, while clipart-A has a nonsemantic shift.”
Energy-Based Open-World Uncertainty Modeling for Confidence Calibration (2021, methodology): “Research on conﬁdence calibration for DNNs has received increasing attention in recent years [14, 27, 19, 23, 27].”
Is Out-of-Distribution Detection Learnable? (2022, background): “[3] Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira.”
A survey of uncertainty in deep neural networks (2021, methodology): “applying the sigmoid function to each logit [105].”
ReAct: Out-of-distribution Detection With Rectified Activations (2021, background): “Early works attempted to improve the OOD uncertainty estimation by proposing the ODIN score [17, 31], OpenMax score [3], and Mahalanobis distance-based score [29].”
ViM: Out-Of-Distribution with Virtual-logit Matching (2022, methodology): “MOS Residual Mahalanobis Energy MaxLogitODIN KL Matching MSP ViM (Ours) 75 80 85 90 95 55 60 65 70 75 80 85 O pe nI m ag eO (A U RO C) ImageNet-O (AUROC) Figure 1.”
Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer (2021, unknown): “Semantic OoD detection is a difficult challenge (Hsu et al., 2020) and we do not have access to any known OoD data to calibrate our confidence.”
Mitigating Neural Network Overconfidence with Logit Normalization (2022, methodology): “In addition, GODIN (Hsu et al., 2020) and Cosine loss (Techa-panurak & Okatani, 2019) adopt cosine similarity for better performance on OOD detection.”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “Enhancement [69], [112], [113], [114], [115], [116], [117], [118], [119], [120], [121], [122], [123], [124], [125]”
Energy-based Out-of-distribution Detection (2020, methodology): “A plethora of recent research has studied the issue of out-of-distribution detection [2,3,14–16,20,23,24,27].”
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection (2022, methodology): “G-ODIN [67] decomposes the posterior to explicitly model the probability of ID-ness.”
On the Importance of Gradients for Detecting Distributional Shifts in the Wild (2021, methodology): “Meanwhile, some works considered covariate shifts in the input space [30, 36], where inputs can be corruption-shifted or domain-shifted [14].”
MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space (2021, methodology): “A plethora of literature has emerged in addressing the problem of OOD detection [3, 14, 16, 24, 27, 29, 35, 4, 20, 31].”