Centroid Networks for Few-Shot Clustering and Unsupervised Few-Shot Classification

By Gabriel Huang, Hugo Larochelle, Simon Lacoste-Julien. 2019

22 citations recorded by the source corpus.

Traditional clustering algorithms such as K-means rely heavily on the nature of the chosen metric or data representation. To get meaningful clusters, these representations need to be tailored to the downstream task (e.g. cluster photos by object category, cluster faces by identity). Therefore, we frame clustering as a meta-learning task, few-shot clustering, which allows us to specify how to cluster the data at the meta-training level, despite the clustering algorithm itself being unsupervised. We propose Centroid Networks, a simple and efficient few-shot clustering method based on learning representations which are tailored both to the task to solve and to its internal clustering module. We also introduce unsupervised few-shot classification, which is conceptually similar to few-shot clustering, but is strictly harder than supervised* few-shot classification and therefore allows direct comparison with existing supervised few-shot classification methods. On Omniglot and miniImageNet, our method achieves accuracy competitive with popular supervised few-shot classification algorithms, despite using *no labels* from the support set. We also show performance competitive with state-of-the-art learning-to-cluster methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks (2020, methodology): “Previous work has associated flat minimizers with good generalization (Huang et al., 2019b).”
Learning to Discover Novel Visual Categories via Deep Transfer Clustering (2019, methodology): “…0.353 71.9% 0.713 LPNMF [4] 22.2% 0.372 43.0% 0.526 LSC [8] 23.6% 0.376 73.3% 0.733 KCL [15] 82.4% 0.889 73.8% 0.750 MCL [16] 83.3% 0.897 74.4% 0.762 Centroid Networks [17] 86.6% - - - DTC 89.0 % 0.949 78.3 % 0.791 noting that Centroid Networks [17] also assumes the clusters to be of uniform size.”
Locality and compositionality in zero-shot learning (2019, background): “Prototypical networks are chosen as they require minimal parameters or hyper-parameters tuning, are well-studied (Huang et al., 2019; Finn et al., 2017), and performance is very close to the state of the art for Imagenet-pretrained benchmarks.”
Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations (2020, background): “Using prototypical networks for ZFS requires minimal parameters or hyper-parameters tuning, is well-studied (Huang et al., 2019; Finn et al., 2017), and performance is very close to the state of the art for Imagenet-pretrained benchmarks.”
Deep Amortized Clustering (2019, methodology): “Centroid Networks (Huang et al., 2019) learn an embedding which is clustered with the Sinkhorn K-means algorithm.”