Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy

By Qing Yu, Kiyoharu Aizawa. 2019

200 citations recorded by the source corpus.

Since deep learning models have been implemented in many commercial applications, it is important to detect out-of-distribution (OOD) inputs correctly to maintain the performance of the models, ensure the quality of the collected data, and prevent the applications from being used for other-than-intended purposes. In this work, we propose a two-head deep convolutional neural network (CNN) and maximize the discrepancy between the two classifiers to detect OOD inputs. We train a two-head CNN consisting of one common feature extractor and two classifiers which have different decision boundaries but can classify in-distribution (ID) samples correctly. Unlike previous methods, we also utilize unlabeled data for unsupervised training and we use these unlabeled data to maximize the discrepancy between the decision boundaries of two classifiers to push OOD samples outside the manifold of the in-distribution (ID) samples, which enables us to detect OOD samples that are far from the support of the ID samples. Overall, our approach significantly outperforms other state-of-the-art methods on several OOD detection benchmarks and two cases of real-world simulation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Detection and Continual Learning of Novel Face Presentation Attacks (2021, methodology): “Our idea is based on enabling the network to identify new attack types as testing samples that are outside the training distribution samples (OTDS) in an embedding space [11, 29, 40, 19, 4].”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “…examples are made use of to modify loss functions and calibrate the output for detection of unknown unknowns (Bell et al., 2015; Lee et al., 2018b; Yu and Aizawa, 2019; Dhamija et al., 2018; Feng et al., 2019), active learning techniques often resort to conditioning their procedure on the entire…”
Anomalous Instance Detection in Deep Learning: A Survey (2020, methodology): “Semi-supervised [24] A two-head CNN consisting of a common feature extractor and two classifiers with different decision boundaries is trained to detect OOD examples”
Semantically Coherent Out-of-Distribution Detection (2021, methodology): “Table 1: Performance of ODIN [7], MCD [14], and Energy-Based OOD (EBO) [15] on DD-OOD/SC-OOD settings.”
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection (2022, methodology): “Starting from OE [22] which encourages a flat or high-entropic prediction on given OOD samples, MCD [24] uses a two-branch network to enlarge branches’ entropic discrepancy on OOD training data.”
CODEs: Chamfer Out-of-Distribution Examples against Overconfidence Issue (2021, methodology): “Recent works improve OOD detection by using the ODIN score [31, 23], Mahalanobis distance [28], energy score [32], ensemble of multiple classifiers [42, 48], residual flow [51], generative models [38], self-supervised learning [20, 35] and gram matrices [39].”
A survey of uncertainty in deep neural networks (2021, background): “strong fundamental tool set for the deployment in real life applications [254], [241], [255], [56].”
ViM: Out-Of-Distribution with Virtual-logit Matching (2022, methodology): “Several methods, including MCD [39], NGC [36] and UDG [37], can utilize external unlabeled noisy data to enhance the OOD detection performances.”
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning (2020, methodology): “This ﬁnding is consistent with prior works [47,36], where they used only two classiﬁers and achieved promising performance.”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “…samples (Hendrycks et al., 2019b; Dhamija et al., 2018) and suppressing OOD feature magnitudes (Dhamija et al., 2018), a follow-up work, MCD (Yu & Aizawa, 2019) uses a network with two branches, between which entropy discrepancy is enlarged for OOD training data.”