Generative-Discriminative Feature Representations for Open-Set Recognition

By Pramuditha Perera, Vlad I. Morariu, Rajiv Ratan Jain, Varun Manjunatha, Curtis Wigington, Vicente Ordonez, Vishal M. Patel. 2020

220 citations recorded by the source corpus.

We address the problem of open-set recognition, where the goal is to determine if a given sample belongs to one of the classes used for training a model (known classes). The main challenge in open-set recognition is to disentangle open-set samples that produce high class activations from known-set samples. We propose two techniques to force class activations of open-set samples to be low. First, we train a generative model for all known classes and then augment the input with the representation obtained from the generative model to learn a classifier. This network learns to associate high classification probabilities both when image content is from the correct class as well as when the input and the reconstructed image are consistent with each other. Second, we use self-supervision to force the network to learn more informative featues when assigning class scores to improve separation of classes from each other and from open-set samples. We evaluate the performance of the proposed method with recent open-set recognition works across three datasets, where we obtain state-of-the-art results.

Subfields: Artificial Intelligence, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

Language-driven Semantic Segmentation (2022, background): “A variety of solutions have been proposed (Zhang et al., 2020b; Liu et al., 2020a; Perera et al., 2020; Zhou et al., 2021) for open-set recognition (Scheirer et al., 2012; Geng et al., 2020).”
Towards Open World Object Detection (2021, background): “Recently, self-supervised learning [46] and unsupervised learning with reconstruction [65] have been explored for open set recognition.”
Learning Placeholders for Open-Set Recognition (2021, methodology): “Traditional generative-based open-set models tend to utilize powerful generative models to mimic novel patterns [22, 26].”
Learning Bounds for Open-Set Learning (2021, background): “These unknown classes can emerge unexpectedly and drastically weaken the performance of existing closed-set algorithms (de O. Cardoso et al., 2017; Dhamija et al., 2018; Perera et al., 2020).”
Multiple Class Novelty Detection Under Data Distribution Shift (2020, background): “Many problems such as countering adversarial/trojan/poison attacks [26,25,8,47], detecting novel categories [42,40,7,37,35,39] and out-of-distribution samples [13,24,10,54] etc. tackle diﬀerent aspects of robustness of CNNs.”
Human Action Recognition and Prediction: A Survey (2018, background): “However, existing open set recognition (OSR) research works mainly focus on image modality (Scheirer et al., 2012, 2014; Zhang & Patel, 2016; Bendale & Boult, 2016; Oza & Patel, 2019; Perera et al., 2020; Chen et al., 2020), except for a few works on videos (Shu et al.”
Learning to Classify With Incremental New Class (2021, unknown): “Since real-world scenarios are complex, where the ratio of seen and unseen differs in diverse tasks, we use openness [27], [56] to represent the complexity of the detection task 1 The openness becomes higher with more unknown classes, which indicates the dataset is more “open.””
OpenGAN: Open-Set Recognition via Open Data Generation (2021, unknown): “MSP MSPc MCdrop GDM OpenMax GOpenMax OSRCI C2AE CROSR RPL Hybrid GDFR NNpix NNfea OpenGAN OpenGAN Dataset [24] [29] [17] [28] [5] [18]∗ [33]∗ [35]∗ [54]∗ [10]∗ [57]∗ [37]∗ [41] [41] -0pix -0fea MNIST .”
Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP (2021, background): “Perera et al. (2020) is a recent hybrid model based on generative-discriminative features.”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “Apart from explicitly forcing discrepancy between known/unknown classes, other methods extract stronger features through an auxiliary task of transformation classification [188], or mutual information maximization between the input image and its latent features [189], etc.”