Towards Open Set Deep Networks
By Abhijit Bendale, Terrance E. Boult. 2015
1,801 citations recorded by the source corpus.
Deep networks have produced significant gains for various visual recognition problems, leading to high impact academic and commercial applications. Recent work in deep networks highlighted that it is easy to generate images that humans would never classify as a particular object class, yet networks classify such images high confidence as that given class - deep network are easily fooled with images humans do not consider meaningful. The closed set nature of deep networks forces them to choose from one of the known classes leading to such artifacts. Recognition in the real world is open set, i.e. the recognition system should reject unknown/unseen classes at test time. We present a methodology to adapt deep networks for open set recognition, by introducing a new model layer, OpenMax, which estimates the probability of an input being from an unknown class. A key element of estimating the unknown probability is adapting Meta-Recognition concepts to the activation patterns in the penultimate layer of the network. Open-Max allows rejection of "fooling" and unrelated open set images presented to the system, OpenMax greatly reduces the number of obvious errors made by a deep network. We prove that the OpenMax concept provides bounded open space risk, thereby formally providing an open set recognition solution. We evaluate the resulting open set deep networks using pre-trained networks from the Caffe Model-zoo on ImageNet 2012 validation data, and thousands of fooling and open set images. The proposed OpenMax model significantly outperforms open set recognition accuracy of basic deep networks as well as deep networks with thresholding of SoftMax probabilities.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Anomaly & Outlier Detection
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Bernt Schiele
- Yoshua Bengio
- Trevor Darrell
- Sergey Levine
- Mohit Bansal
- Dacheng Tao
- Percy Liang
- Quoc V. Le
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Ming-Hsuan Yang
- Bernt Schiele
- Xiaogang Wang
- Andrew Zisserman
- Jitendra Malik
In Anomaly & Outlier Detection
Citations in the corpus
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, neutral): “4.1.1) [6, 7, 11, 92, 93].”
- Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, methodology): “[6] A.”
- Model-Reuse Attacks on Deep Learning Systems (2018, background): “One may suggest using incremental learning [15, 48], which retrains an existing model to accommodate new data, or open-set learning, which extends a given model to new classes during inference [4, 51].”
- On the (Statistical) Detection of Adversarial Examples (2017, methodology): “sarial examples throughout training but rather use adversarial examples precomputed on a different model before training. Nguyen et al [24] have introduced an outlier class before. Also Bendale et al [3] propse open networks, that are not confident in their classification all over the feature space.”
- Is Deep Learning Safe for Robot Vision? Adversarial Examples Against the iCub Humanoid (2017, methodology): “5-class classification (based on the combination of anomaly detectors and two-class classifiers) [3] to open-set recognition techniques [23, 2].”
- Efficient Defenses Against Adversarial Attacks (2017, background): “crafted noise) are interpreted by the model as natural examples with high confidence [2, 31].”
- Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics (2016, background): “We note that this issue was also pointed out by [2] in a different manner and they proposed a solution in the OpenMax classifier, which we compare against in the experiments.”
- Towards Open World Object Detection (2021, background): “Bendale and Boult [4] identified unknowns in the feature space of deep networks and used a Weibull distribution to estimate the set risk (called OpenMax classifier).”
- Out-of-Domain Detection for Natural Language Understanding in Dialog Systems (2019, methodology): “Various methods have been proposed to improve the OOD detection performance of neural NLU models [6]–[10], and most of them follow a threshold-based protocol.”
- Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem (2018, background): “Another approach is the introduction of a rejection option into the classifier [32, 4], in order to avoid decisions the classifier is not certain about.”
- DOC: Deep Open Classification of Text Documents (2017, methodology): “The work closest to ours is that in (Bendale and Boult, 2016), which leverages an algorithm called OpenMax to add the rejection capability by utilizing the logits that are trained via closed-world softmax function.”
- Lightweight Probabilistic Deep Networks (2018, background): “Even worse, predictive distributions of common classification models relying on the softmax are not well calibrated and tend to be overconfident [4, 13, 19].”
- The limits and potentials of deep learning for robotics (2018, background): “This is not possible when unknown classes are expected at testing time [12].”
- Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017, background): “hand, [62] introduces a newmodel layer to the deep netwhich estimates the probability of an input being from an unknown”
- Zero-Shot Learning — The Good, the Bad and the Ugly (2017, background): “On the other hand, [5] introduces a new model layer to the deep net which estimates the probability of an input being from an unknown class and [34] proposes a novelty detection mechanism.”
- A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, methodology): “Prior works that address open set recognition or in general application of machine learning algorithms in an open world have argued that prediction on previously unseen unknown classes results in inevitable misprediction [15], [67], [127], [128], [131].”
- Large-Scale Long-Tailed Recognition in an Open World (2019, methodology): “For the open-set setting, the F-measure is also reported for a balanced treatment of precision and recall following [2].”
- Learning Placeholders for Open-Set Recognition (2021, methodology): “[3] first proposed to replace the softmax layer in the network with OpenMax, which calibrates the output probability with Weibull distribution.”