Generalized Category Discovery

By Sagar Vaze, Kai Han, Andrea Vedaldi, Andrew Zisserman. 2022

325 citations recorded by the source corpus.

In this paper, we consider a highly general image recognition setting wherein, given a labelled and unlabelled set of images, the task is to categorize all images in the unlabelled set. Here, the unlabelled images may come from labelled classes or from novel ones. Existing recognition methods are not able to deal with this setting, because they make several restrictive assumptions, such as the unlabelled instances only coming from known – or unknown – classes, and the number of unknown classes being known a-priori. We address the more unconstrained setting, naming it ‘Generalized Category Discovery’, and challenge all these assumptions. We first establish strong baselines by taking state-of-the-art algorithms from novel category discovery and adapting them for this task. Next, we propose the use of vision transformers with contrastive representation learning for this open-world setting. We then introduce a simple yet effective semi-supervised k-means method to cluster the unlabelled data into seen and unseen classes automatically, substantially outperforming the baselines. Finally, we also propose a new approach to estimate the number of classes in the unlabelled data. We thoroughly evaluate our approach on public datasets for generic object classification and on fine-grained datasets, leveraging the recent Semantic Shift Benchmark suite. Code: https://www.robots.ox.ac.uk/~vgg/research/gcd

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Kai Han

In Signal Processing

Citations in the corpus

Novel Class Discovery without Forgetting (2022, background): “[52] extends NCD to a generalized setting where the unlabeled instances may come from both old and new classes, while still requiring access to labeled and unlabeled instances jointly.”
Deep ViT Features as Dense Visual Descriptors (2021, methodology): “Recent works follow this observation and utilize these features for object discovery [48, 54], semantic segmentation [20] and category discovery [52].”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “For example,apartfromnovelmethods,OSRresearchalsoshows that a good classiﬁer (Vaze et al., 2022b) in the close-set is critical to OSR performance, which should also applicable to OOD detection tasks.”