Cluster Alignment With a Teacher for Unsupervised Domain Adaptation

By Zhijie Deng, Yucen Luo, Jun Zhu. 2019

246 citations recorded by the source corpus.

Deep learning methods have shown promise in unsupervised domain adaptation, which aims to leverage a labeled source domain to learn a classifier for the unlabeled target domain with a different distribution. However, such methods typically learn a domain-invariant representation space to match the marginal distributions of the source and target domains, while ignoring their fine-level structures. In this paper, we propose Cluster Alignment with a Teacher (CAT) for unsupervised domain adaptation, which can effectively incorporate the discriminative clustering structures in both domains for better adaptation. Technically, CAT leverages an implicit ensembling teacher model to reliably discover the class-conditional structure in the feature space for the unlabeled target domain. Then CAT forces the features of both the source and the target domains to form discriminative class-conditional clusters and aligns the corresponding clusters across domains. Empirical results demonstrate that CAT achieves state-of-the-art results in several unsupervised domain adaptation scenarios.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, methodology): “Prototypes-based regularizing terms gained a great interest and, in particular, have been largely used in the literature of few-shot learning [15, 63, 60], to learn prototypical representations of each category, and domain adaptation, to enforce orthogonality [50, 65] or centroid matching [70, 12].”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, unknown): “In the same spirit of encouraging consistent outputs under perturbations, various UDA approaches apply input transformations or model ensembling to simulate variations in input or model space [39], [248], [249], [250].”
Harmonizing Transferability and Discriminability for Adapting Object Detectors (2020, background): “, prototype alignment [58, 5, 37, 10] or entropy regularization [64, 47, 8]) would be no longer applicable.”
Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation (2022, methodology): “Prototype-based approaches were initially proposed for few-shot learning [8, 35] and domain adaptation [6, 29, 38].”
Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer (2020, methodology): “For vanilla unsupervisedDA in digit recognition, we compare SHOT with ADDA [5], ADR [56], CDAN [42], CyCADA [8], CAT [98], SWD [99] and STAR [100]; for object recognition, we compare ours with DANN [24], DAN [4], SAFN [82], BSP [101], MDD [102], TransNorm [55], DSBN [25], BNM [103] and GVB-GD [104].”
Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering (2020, methodology): “We note that quite a few recent UDA methods [13, 27, 41, 51] consider clustering of target data as well; however, they still do explicit feature alignment between the two domains via alignment of cluster centers/samples, thus prone to the aforementioned risk of damaged intrinsic target discriminatio”
Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation (2020, methodology): “In practice, UDA methods have been applied to many real-world problems, such as object recognition [Agresti et al. , 2019; Deng et al. , 2019; Zhao et al. , 2019].”
Cross-Domain First Person Audio-Visual Action Recognition through Relative Norm Alignment (2021, background): “, the maximum mean discrepancy (MMD) in [36], and adversarial-based methods [16, 53], often leveraging a gradient reversal layer (GRL) [19].”
Domain-Adaptive Few-Shot Learning (2020, background): “Given that they are mainly designed for distribution alignment across domains (recently focusing on per-class alignment [47, 32, 5, 25, 46]), they are intrinsically unsuited for FSL whereby the target classes are completely different from the source ones: either global or per-class distribution alig”
Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (2020, methodology): “For mutli-source and multi-target DA, we compare with DCTN (Xu et al., 2018), MCD (Saito et al., (Long et al., 2018; Deng et al., 2019; Xu et al., 2019; Peng et al., 2019a).”
A Survey of Unsupervised Deep Domain Adaptation (2018, unknown): “[58] combine target discriminative methods with selfensembling.”
Adversarial Domain Adaptation With Prototype-Based Normalized Output Conditioner (2020, methodology): “…with the ResNet-50 [44] 78.0 65.5 91.5 74.8 91.2 83.9 80.7 DANN [20] 87.0 74.3 96.2 75.0 91.5 86.0 85.0 CDAN [14] 90.5 74.5 97.0 76.7 93.5 90.6 87.1 CDAN+E [14] 91.3 74.2 97.7 77.7 94.3 90.7 87.7 MADA [13] 88.8 75.2 96.0 75.0 92.2 87.9 85.8 iCAN [67] 89.9 78.5 94.7 79.5 92.0 89.7 87.4 CAT [66] 91.”