Automatically Discovering and Learning New Visual Categories with Ranking Statistics

By Kai Han, Sylvestre-Alvise Rebuffi, Sébastien Ehrhardt, Andrea Vedaldi, Andrew Zisserman. 2020

227 citations recorded by the source corpus.

We tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. This setting is similar to semi-supervised learning, but significantly harder because there are no labelled examples for the new classes. The challenge, then, is to leverage the information contained in the labelled images in order to learn a general-purpose clustering model and use the latter to identify the new classes in the unlabelled data. In this work we address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labeled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use rank statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data. We evaluate our approach on standard classification benchmarks and outperform current methods for novel category discovery by a significant margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Towards Label-Efficient Incremental Learning: A Survey (2023, background): “However, existing incremental learners currently has no novel object discovery capacity [Han et al. , 2020], as all the objects are at least partially labeled.”
Novel Class Discovery without Forgetting (2022, methodology): “– When compared with prominent class-discovery methods [16,19,60] adapted to our proposed setting, our methodology achieves improved class-discovery performance with significantly less forgetting.”
Class-incremental Novel Class Discovery (2022, methodology): “We split the data sets into the old and new classes following the existing NCD and iNCD works [15,36,29].”
OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World (2020, methodology): “Following [7], we conduct the experiment on the setting where the number of classes in unlabeled data is known.”
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation (2021, methodology): “Novel category discovery is a relatively new problem setting recently formalized by [20, 18] with the task being automatically discovering new object categories in the unlabelled data by making use of a labeled dataset containing different but relevant object classes.”
Continuum: Simple Management of Complex Continual Learning Scenarios (2021, background): “However, recent works challenge some of those common assumptions such as fixed label [6, 2], uncorrelated data stream [5, 42], labeled new data [16], or even no metadata about tasks and unavailability of information about the future [11].”
Insights from the Future for Continual Learning (2020, background): “Han et al. [15] proposed a setting where the training samples from all classes are present from the beginning, but the labels become available incrementally.”
A Unified Objective for Novel Class Discovery (2021, methodology): “We evaluate the performance of our method on three well-established NCD benchmarks following [8, 7], i.”
Generalized Category Discovery (2022, methodology): “AutoNovel (aka Rankstats) [17, 18] tackles the NCD problem with a three stage method.”
Spacing Loss for Discovering Novel Categories (2022, methodology): “RS [11, 12], NCL [45] and OpenMix [46] first use RotNet [22] to self-supervise on the labeled and unlabeled data.”
Residual Tuning: Toward Novel Category Discovery Without Labels (2022, unknown): “By comparing with DTC [9] and AutoNovel [21], we observe ResTune enables to generate a more discriminative boundary between the two sets of labeled and unlabeled classes.”
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data (2021, methodology): “The choice of ResNet-18 for image datasets is to follow [18, 16] for fair comparison.”
Novel Class Discovery in Semantic Segmentation (2021, background): “[9] introduce self-supervised learning to bootstrap feature representation and ranking statistics to explore similarities within unlabeled samples.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “These more realistic setups pose multiple new challenges, including con-ﬁdence calibration of DNN for OOD detection [223], [223], [225], [226], imbalanced class distribution caused by real-world long-tailed unlabeled data [227], [228], and discovery of unseen classes[224],[229],[230].”