Learning To Classify Images Without Labels

By Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Marc Proesmans, Luc Van Gool. 2020

677 citations recorded by the source corpus.

Can we automatically group images into semantically meaningful clusters when ground-truth annotations are absent? The task of unsupervised image classification remains an important, and open challenge in computer vision. Several recent approaches have tried to tackle this problem in an end-to-end fashion. In this paper, we deviate from recent works, and advocate a two-step approach where feature learning and clustering are decoupled. First, a self-supervised task from representation learning is employed to obtain semantically meaningful features. Second, we use the obtained features as a prior in a learnable clustering approach. In doing so, we remove the ability for cluster learning to depend on low-level features, which is present in current end-to-end learning approaches. Experimental evaluation shows that we outperform state-of-the-art methods by large margins, in particular +26.6% on CIFAR10, +25.0% on CIFAR100-20 and +21.3% on STL10 in terms of classification accuracy. Furthermore, our method is the first to perform well on a large-scale dataset for image classification. In particular, we obtain promising results on ImageNet, and outperform several semi-supervised learning methods in the low-data regime without the use of any ground-truth annotations. The code is made publicly available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Grafit: Learning fine-grained image representations with coarse labels (2020, methodology): “In unsupervised and self-supervised approaches [9, 10, 22, 25, 34, 56] the model is trained on unlabeled data.”
Hard Negative Mixing for Contrastive Learning (2020, methodology): “Although some care about the actual clustering performance on the training dataset [22, 53, 88], others further use the clustering losses as means for learning representations that generalize [8–10, 84, 82, 5].”
Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos (2021, methodology): “In contrast, instead of keeping the clustering step independent of the representation learning phase, more recent techniques jointly learn visual embeddings and cluster assignments [5, 6, 11, 41].”
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, background): “Studies [193], [194], [195] have shown that the category-level contrastive learning signal has significant advantages over the instance-level contrastive learning signal”
Self-Supervised Features Improve Open-World Learning (2021, background): “com/Vastlab/SSFiOWL ignored the importance of feature representations, which has been studied in broader settings [31, 44, 19, 33].”
No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems (2020, methodology): “of classes, either using clustering techniques [6] or in an end-to-end approach [25, 18].”
iBOT: Image BERT Pre-Training with Online Tokenizer (2021, methodology): “We compare our methods to SimCLRv2 (Chen et al., 2020b), Self-label (Asano et al., 2020), InfoMin (Tian et al., 2020), and SCAN (Van Gansbeke et al., 2020).”
Neighborhood Contrastive Learning for Novel Class Discovery (2021, background): “To this end, many shallow [1, 23, 34] and deep learning based methods [6, 9, 24, 29, 32, 33, 36] have been proposed.”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “Since computing all the pairwise comparisons on a large dataset is challenging, some clustering-based methods turn to discriminate between groups of images with similar features instead of individual images [2, 5, 12, 13, 64, 104, 119, 121, 122].”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “• Self supervised learning (SSL) [243], [490] is an important subset of unsupervised learning for generating output labels from different data objects by find-”
Wavelet Convolutions for Large Receptive Fields (2024, methodology): “We construct a small-scale scalability analysis for the task of classification on ImageNet-50/100/200 [46,55], which are subsets of ImageNet [10] containing 50/100/200 classes, respectively.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, methodology): “These visual variations can be intra-instance variations such as different views of the same instance [125], [126], [127], [128], [129], or inter-instance variations between neighbor instances [130], [131] or across a group of instances [132], [133], [134].”
Partition-Guided GANs (2021, background): “However, this may result in degenerated clusters where one partition contains most of the data [6, 74].”
Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorization (2020, methodology): “Explainability for Self-Supervised Models: We use three state-of-the-art SSL models: ResNet-50 trained with either SCAN (Van Gansbeke et al. 2020) or SeLa (Asano, Rupprecht, and Vedaldi 2019b), and an Alexnet by Asano, Rupprecht, and Vedaldi (2019a), which we denote as RotNet.”
Unsupervised Continual Learning Via Pseudo Labels (2021, methodology): “Note that the classiﬁcation accuracy used in supervised setting is consistent with cluster accuracy and is widely used for performance comparison in unsupervised case as in [Van Gans-beke et al. , 2020].”
Generalized Source-free Domain Adaptation (2021, methodology): “This is similar to methods in unsupervised learning [43, 10, 50, 40] or domain adaptation [29].”
Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning (2022, methodology): “C Pseudo labels (Van Gansbeke et al., 2020) uses a contrastive learning setup similar to SimCLR ( ?”
A Unified Objective for Novel Class Discovery (2021, background): “Moreover, differently from common clustering [1, 25] scenarios, in an NCD framework, labeled data can be utilized at training time, and the challenge consists in transferring the supervised knowledge on the known classes to improve clustering of the unknown ones.”