Open-Set Recognition: A Good Closed-Set Classifier is All You Need

By Sagar Vaze, Kai Han, Andrea Vedaldi, Andrew Zisserman. International Conference on Learning Representations, 2021

625 citations recorded by the source corpus.

The ability to identify whether or not a test sample belongs to one of the semantic classes in a classifier's training set is critical to practical deployment of the model. This task is termed open-set recognition (OSR) and has received significant attention in recent years. In this paper, we first demonstrate that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes. We find that this relationship holds across loss objectives and architectures, and further demonstrate the trend both on the standard OSR benchmarks as well as on a large-scale ImageNet evaluation. Second, we use this correlation to boost the performance of a maximum logit score OSR 'baseline' by improving its closed-set accuracy, and with this strong baseline achieve state-of-the-art on a number of OSR benchmarks. Similarly, we boost the performance of the existing state-of-the-art method by improving its closed-set accuracy, but the resulting discrepancy with the strong baseline is marginal. Our third contribution is to present the 'Semantic Shift Benchmark' (SSB), which better respects the task of detecting semantic novelty, in contrast to other forms of distribution shift also considered in related sub-fields, such as out-of-distribution detection. On this new evaluation, we again demonstrate that there is negligible difference between the strong baseline and the existing state-of-the-art. Project Page: https://www.robots.ox.ac.uk/~vgg/research/osr/

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/ba4dc8bdf6f1b9756def3a9b6a52fd797d13f4d4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ziwei Liu
2. Kaiyang Zhou
3. Yixuan Li
4. Andrew Zisserman
5. Andrea Vedaldi
6. Bo Li
7. Jingkang Yang
8. Yu Qiao

Citations in the corpus

• Retrieval-Augmented Generation for Large Language Models: A Survey (2023, background): “This division ensures a thorough evaluation of both the quality of context provided and the quality of content produced.”
• Generalized Out-of-Distribution Detection: A Survey (2021, positive): “For example,apartfromnovelmethods,OSRresearchalsoshows that a good classifier (Vaze et al., 2022b) in the close-set is critical to OSR performance, which should also applicable to OOD detection tasks.”
• OpenOOD: Benchmarking Generalized Out-of-Distribution Detection (2022, background): “The common practice for building OSR benchmarks is to divide the categories of existing datasets into two parts, called closed and open set [40, 41].”
• Generalized Category Discovery (2022, methodology): “Results on the FGVC-Aircraft [31] splits from the Semantic Shift Benchmark suite [45].”