Learning to cluster in order to Transfer across domains and tasks

By Yen-Chang Hsu, Zhaoyang Lv, Zsolt Kira. 2017

246 citations recorded by the source corpus.

This paper introduces a novel method to perform transfer learning across domains and tasks, formulating it as a problem of learning to cluster. The key insight is that, in addition to features, we can transfer similarity information and this is sufficient to learn a similarity function and clustering network to perform both domain adaptation and cross-task transfer learning. We begin by reducing categorical information to pairwise constraints, which only considers whether two instances belong to the same class or not. This similarity is category-agnostic and can be learned from data in the source domain using a similarity network. We then present two novel approaches for performing transfer learning using this similarity function. First, for unsupervised domain adaptation, we design a new loss function to regularize classification with a constrained clustering loss, hence learning a clustering network with the transferred similarity metric generating the training inputs. Second, for cross-task learning (i.e., unsupervised clustering with unseen categories), we propose a framework to reconstruct and estimate the number of semantic clusters, again using the clustering network. Since the similarity network is noisy, the key is to use a robust clustering algorithm, and we show that our formulation is more robust than the alternative constrained and unconstrained clustering approaches. Using this method, we first show state of the art results for the challenging cross-task problem, applied on Omniglot and ImageNet. Our results show that we can reconstruct semantic clusters with high accuracy. We then evaluate the performance of cross-domain transfer using images from the Office-31 and SVHN-MNIST tasks and present top accuracy on both datasets. Our approach doesn't explicitly deal with domain discrepancy. If we combine with a domain adaptation loss, it shows further improvement.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

Set2Graph: Learning Graphs From Sets (2020, background): “…learning Previous models suggested for learning pairwise relations in sets were mostly of the form H 2 ( X ) = m ( φ ( x i ) , φ ( x j )) (Hsu et al., 2017; Bromley et al., 1994; Simo-Serra et al., 2015; Zagoruyko & Komodakis, 2015; Bell & Bala, 2015; Ahmed et al., 2015; Vo & Hays, 2016).”
Novel Class Discovery without Forgetting (2022, methodology): “Earlier methods like MCL [25] and KCL [24] for general transfer learning across domains and tasks meta-learn a binary similarity function from the labeled data and use it to discover classes in the unlabeled data.”
Class-incremental Novel Class Discovery (2022, methodology): “Most of the proposed NCD solutions rely on stage-wise [19,20,16] or joint [15,36,11] learning schemes on the labelled and the unlabelled data, with the assumption that structures discovered on the labelled images could be leveraged as a proxy supervision on the unlabelled images.”
Large-Scale Long-Tailed Recognition in an Open World (2019, methodology): “We follow [21] on class structure analysis and adopt discriminative centroids as the basic building block.”
Novel Class Discovery in Semantic Segmentation (2021, methodology): “[13].”
A Survey of Unsupervised Deep Domain Adaptation (2018, background): “CCN++[101]∗ 89.”
Clustering-Oriented Representation Learning with Attractive-Repulsive Loss (2018, methodology): “This includes pairwise-based methods (Hsu and Kira 2016; Fogel et al. 2018; Hsu, Lv, and Kira 2018), and triplet-based methods (Norouzi, Fleet, and Salakhutdinov 2012; Schroff, Kalenichenko, and Philbin 2015; He et al. 2018); all of these require sophisticated methods for sampling training data,…”
Mapping single-cell data to reference atlases by transfer learning (2021, background): “Reference atlases provide an opportunity to radically change how we currently analyze single-cell datasets: by learning from the appropriate reference, we could auto-mate annotation of new datasets and easily perform comparative analyses across tissues, species and disease conditions.”
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data (2021, methodology): “For example, after the enhancement, the ACC of KCL is increased to 73.9%/57.4%/ 74.3% on CIFAR10/CIFAR-100/ImageNet, while the results of our method are 93.4%/76.4%/86.7%, demonstrating the effectiveness of our approach on image benchmarks.”
Multi-class Classification without Multi-class Labels (2019, methodology): “, 2012), and Kullback–Leibler divergence based contrastive loss (KCL) (Hsu & Kira, 2016; Hsu et al., 2018).”
A Unified Objective for Novel Class Discovery (2021, methodology): “tering step on the unlabeled data [8, 10, 11].”
Generalized Category Discovery (2022, methodology): “Earlier methods that could be applied to this problem include KCL [21] and MCL [22], both of which maintain two models trained with labelled data and unlabelled data respectively, for general task transfer learning.”
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation (2021, methodology): “It can be seen that our method significantly outperforms k-means, KCL [25], MCL [26], and DTC [20].”
Automatically Discovering and Learning New Visual Categories with Ranking Statistics (2020, methodology): “Following Hsu et al. (2018; 2019), for OmniGlot, each of the 20 alphabets in evaluation set (with 20–47 categories, 659 characters/class) is used as unlabelled data, and all the 30 alphabets in background set are used as labelled set (964 characters/class).”
Domain Generalization via Model-Agnostic Learning of Semantic Features (2019, methodology): “Leveraging distance metrics of feature vectors is another method [20, 34].”
Residual Tuning: Toward Novel Category Discovery Without Labels (2022, unknown): “1) “Isolated Training” methods include KCL [14], MCL [15], DTC [9], and our ResTune.”
Temporal Attentive Alignment for Large-Scale Video Domain Adaptation (2019, methodology): “ply TA(3)N to different cross-domain video tasks, including video captioning, segmentation, and detection; 2) we would like to extend these methods to the open-set setting [1, 39, 33, 15], which has different categories between source and target domains.”
Neighborhood Contrastive Learning for Novel Class Discovery (2021, methodology): “We compare the proposed approach with one classical method (k-means [23]) and four state-of-the-art methods (i.e., KCL [16], MCL [17], DTC [13] and RS [12]).”