C2AE: Class Conditioned Auto-Encoder for Open-Set Recognition

By Poojan Oza, Vishal M. Patel. Computer Vision and Pattern Recognition, 2019

412 citations recorded by the source corpus.

Models trained for classification often assume that all testing classes are known while training. As a result, when presented with an unknown class during testing, such closed-set assumption forces the model to classify it as one of the known classes. However, in a real world scenario, classification models are likely to encounter such examples. Hence, identifying those examples as unknown becomes critical to model performance. A potential solution to overcome this problem lies in a class of learning problems known as open-set recognition. It refers to the problem of identifying the unknown classes during testing, while maintaining performance on the known classes. In this paper, we propose an open-set recognition algorithm using class conditioned auto-encoders with novel training and testing methodologies. In this method, training procedure is divided in two sub-tasks, 1. closed-set classification and, 2. open-set identification (i.e. identifying a class as known or unknown). Encoder learns the first task following the closed-set classification training pipeline, whereas decoder learns the second task by reconstructing conditioned on class identity. Furthermore, we model reconstruction errors using the Extreme Value Theory of statistical modeling to find the threshold for identifying known/unknown class samples. Experiments performed on multiple image classification datasets show that the proposed method performs significantly better than the state of the art methods. The source code is available at: github.com/otkupjnoz/c2ae.

Subfields: Artificial Intelligence, Media Technology

View Paper: https://www.semanticscholar.org/paper/967d532a66dab7edcb818b0f9dc59fe8da7dc171

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Media Technology

Citations in the corpus

Generative-Discriminative Feature Representations for Open-Set Recognition (2020, methodology): “The method proposed in [19] used a class conditioned generator to learn a representation that preserves only known-class samples.”
Learning Bounds for Open-Set Learning (2021, methodology): “CGDL (Sun et al., 2020), C2AE (Oza & Patel, 2019) and Counterfactual (Neal et al., 2018) are the representative works based on generative models.”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, neutral): “• iCarl [25] •MRGAN [21] •VCL [20] •VGR [19] •VASE [18] • FearNet [27] • LLRNN [100] • LLGAN [22] • CAP [131] •OpenMax [67] •Mahalanobis [132] •OWR-Survey [15] • CROSR [133] • C2AE [156] • Latent based EVT [40]”
OCGAN: One-Class Novelty Detection Using GANs With Constrained Latent Representations (2019, background): “Different from other machine learning tasks [12],[15], in one-class novelty detection, examples of only a single class are observed at training time.”
Adversarial Reciprocal Points Learning for Open Set Recognition (2021, methodology): “These generative methods [20], [24], [39] consider using decoder to optimize the deep feature space, but they do not pay attention to the characteristics of unknown distribution in deep feature space.”
Hybrid Models for Open Set Recognition (2020, methodology): “rformance of ow models. We perform extensive experiments on various benchmarks including MNIST, SVHN, CIFAR10 and TinyImageNet. The proposed OpenHybrid model outperforms both state-of-the-art methods [2,7,22,24,37] and hybrid model baselines [20,10] in these benchmarks. We further compare our method”
Conditional Gaussian Distribution Learning for Open Set Recognition (2020, background): “The reconstruction errors of input from known classes are commonly smaller than that of unknown classes [24].”
Generalized Out-of-Distribution Detection: A Survey (2021, methodology): “By fixing the visual encoder obtained from standard multiclass training to maintain ID classification performance, C2AE trains a decoder conditioned on label vectors and estimates the reconstructed images using EVT to distinguish unknown classes [183].”
Generalized Category Discovery (2022, methodology): “Several methods have been proposed to train models such that images with large reconstruction error are regarded as open-set samples [31, 38, 45].”
Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP (2021, methodology): “In most papers about open set detection or open world classification, seen and unseen classes in evaluation are often two splits of the same dataset (Fei and Liu 2016; Shu, Xu, and Liu 2017; Bendale and Boult 2016; Oza and Patel 2019; Perera et al. 2020; Miller et al. 2021; Pernici et al. 2018; Xu e”
Recent Advances in Open Set Recognition: A Survey (2018, methodology): “Using class conditioned auto-encoders with novel training and testing methodology, Oza and Patel [87] proposed C2AE model”
Learning Placeholders for Open-Set Recognition (2021, methodology): “We compare to the SOTA methods, i.e ., Softmax, OpenMax [3], G-OpenMax [6], OSRCI [22], C2AE [24], CROSR [36], GFROSR [26]: Softmax : utilizes the highest softmax probability as the conﬁdence score for detection; OpenMax : replaces softmax layer with OpenMax and cal-ibrates the conﬁdence to predict…”
CMG: A Class-Mixed Generation Approach to Out-of-Distribution Detection (2022, neutral): “So far, many approaches have been proposed to solve this problem, from distance-based meth-ods [2, 3, 12, 20], to generative models [60, 38, 41, 36] and self-supervised learning methods [4, 11, 17, 19].”
OpenGAN: Open-Set Recognition via Open Data Generation (2021, positive): “MNIST/CIFAR/SVHN/TinyImageNet are widely used in the open-set literature, and we follow the literature to experiment with these datasets [33, 35].”