Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation

By Bingchen Zhao, Kai Han. 2021

153 citations recorded by the source corpus.

In this paper, we tackle the problem of novel visual category discovery, i.e., grouping unlabelled images from new classes into different semantic partitions by leveraging a labelled dataset that contains images from other different but relevant categories. This is a more realistic and challenging setting than conventional semi-supervised learning. We propose a two-branch learning framework for this problem, with one branch focusing on local part-level information and the other branch focusing on overall characteristics. To transfer knowledge from the labelled data to the unlabelled, we propose using dual ranking statistics on both branches to generate pseudo labels for training on the unlabelled data. We further introduce a mutual knowledge distillation method to allow information exchange and encourage agreement between the two branches for discovering new categories, allowing our model to enjoy the benefits of global and local features. We comprehensively evaluate our method on public benchmarks for generic object classification, as well as the more challenging datasets for fine-grained visual recognition, achieving state-of-the-art performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Novel Class Discovery without Forgetting (2022, background): “This has been further extended by Zhao and Han [59] to further take local spatial information into account.”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “Novelty Discovery Novelty discovery [93], [94], [95], [96], [97] requires all observations are given in advance as outlier detection does.”
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data (2021, background): “performance by augmenting the training samples by mixing the labelled and unlabelled data [52], aggregating pseudopositive pairs with contrastive learning [51], designing a unified objective [12], and leveraging local part information [50].”
Generalized Category Discovery (2022, background): “Zhao and Han [51] propose a model with two branches, one for global feature learning and the other for local feature learning, such that dual ranking statistics and mutual learning are conducted with these two branches for better representation learning and new class discovery.”
A Unified Objective for Novel Class Discovery (2021, background): “Latter, many NCD works [7, 14, 29, 30, 31] are designed following the two-step training strategy.”
Novel Class Discovery in Semantic Segmentation (2021, background): “[35] further improve the pipeline by dual ranking statistics and mutual knowledge distillation.”
Spacing Loss for Discovering Novel Categories (2022, methodology): “Following existing NCD methods [10, 11, 13, 15, 16, 44, 45], we define splits on CIFAR-10 and CIFAR-100 to evaluate the efficacy of our method.”