Learning to Discover Novel Visual Categories via Deep Transfer Clustering

By Kai Han, Andrea Vedaldi, Andrew Zisserman. 2019

392 citations recorded by the source corpus.

We consider the problem of discovering novel object categories in an image collection. While these images are unlabelled, we also assume prior knowledge of related but different image classes. We use such prior knowledge to reduce the ambiguity of clustering, and improve the quality of the newly discovered classes. Our contributions are twofold. The first contribution is to extend Deep Embedded Clustering to a transfer learning setting; we also improve the algorithm by introducing a representation bottleneck, temporal ensembling, and consistency. The second contribution is a method to estimate the number of classes in the unlabelled data. This also transfers knowledge from the known classes, using them as probes to diagnose different choices for the number of classes in the unlabelled subset. We thoroughly evaluate our method, substantially outperforming state-of-the-art techniques in a large number of benchmarks, including ImageNet, OmniGlot, CIFAR-100, CIFAR-10, and SVHN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Novel Class Discovery without Forgetting (2022, methodology): “DTC [20] formalized the problem of Novel Class Discovery and introduced a method based on Deep Embedded Clustering [55] for NCD, by pre-training it on the labeled data followed by learning-based clustering.”
Class-incremental Novel Class Discovery (2022, methodology): “Specifically, in ResTune, the old head is trained with the objective of LwF [27], whereas the new head is trained with a modified DTC [14] objective.”
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation (2021, methodology): “It can be seen that our method significantly outperforms k-means, KCL [25], MCL [26], and DTC [20].”
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning (2023, background): “There are also several papers that study novel class discovery [18], which is defined as discovering the hidden classes in the detected novel or OOD instances.”
No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems (2020, background): “Further, the fact that work such as [20] directly uses neural representation clustering to estimate the presence of novel classes in a given dataset provides an empirical basis for our approach, which uses a model trained with ERM to approximately identify unlabeled subclasses within each superclass”
Automatically Discovering and Learning New Visual Categories with Ranking Statistics (2020, methodology): “Since we have a relatively larger number of labelled classes in these two datasets, we follow (Han et al., 2019) and use metric learning on the labelled classes to pre-train the feature extractor, instead of the self-supervised learning.”
A Review of Deep Transfer Learning and Recent Advancements (2022, methodology): “[21] 2019 Learning to Discover Novel Visual Categories via Deep Transfer Clustering Image no Fine-tuning yes NA Vision”
Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “Novelty Discovery Novelty discovery [93], [94], [95], [96], [97] requires all observations are given in advance as outlier detection does.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “These more realistic setups pose multiple new challenges, including confidence calibration of DNN for OOD detection [273], [273], [275], [276], [277], imbalanced class distribution caused by realworld long-tailed distributed unlabeled data [278], [279], and discovery of unseen classes in unlabeled d”
Residual Tuning: Toward Novel Category Discovery Without Labels (2022, neutral): “1) “Isolated Training” methods include KCL [14], MCL [15], DTC [9], and our ResTune.”
Self-Supervised Features Improve Open-World Learning (2021, background): “Same as [23, 22], this approach also conducts discovery and recognition of unknown classes just once.”
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data (2021, methodology): “The choice of ResNet-18 for image datasets is to follow [18, 16] for fair comparison.”