Co-regularized Alignment for Unsupervised Domain Adaptation

By Abhishek Kumar, Prasanna Sattigeri, Kahini Wadhawan, Leonid Karlinsky, Rogério Schmidt Feris, William T. Freeman, Gregory W. Wornell. 2018

167 citations recorded by the source corpus.

Deep neural networks, trained with large amount of labeled data, can fail to generalize well when tested with examples from a target domain whose distribution differs from the training data distribution, referred as the source domain. It can be expensive or even infeasible to obtain required amount of labeled data in all possible domains. Unsupervised domain adaptation sets out to address this problem, aiming to learn a good predictive model for the target domain using labeled examples from the source domain but only unlabeled examples from the target domain. Domain alignment approaches this problem by matching the source and target feature distributions, and has been used as a key component in many state-of-the-art domain adaptation methods. However, matching the marginal feature distributions does not guarantee that the corresponding class conditional distributions will be aligned across the two domains. We propose co-regularized domain alignment for unsupervised domain adaptation, which constructs multiple diverse feature spaces and aligns source and target distributions in each of them individually, while encouraging that alignments agree with each other with regard to the class predictions on the unlabeled target examples. The proposed method is generic and can be used to improve any domain adaptation method which uses domain alignment. We instantiate it in the context of a recent state-of-the-art method and observe that it provides significant performance improvements on several domain adaptation benchmarks.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “, 2018) and coercing domain invariance via adversarial learning (or further extensions) (Ganin et al., 2016; Bousmalis et al., 2017; Tzeng et al., 2017; Xie et al., 2018; Hoffman et al., 2018; Long et al., 2018; Zhao et al., 2018b; Kumar et al., 2018; Li et al., 2018b; Zhao et al., 2018a; Schoenauer”
SpotTune: Transfer Learning Through Adaptive Fine-Tuning (2018, background): “Recently, transfer learning based on deep neural networks has received significant attention in the community [12, 6, 7, 24, 13].”
Universal Source-Free Domain Adaptation (2020, background): “Unsupervised Domain Adaptation (DA) algorithms seek to minimize this discrepancy without accessing the target label information, either by learning a domain invariant feature representation [26, 21, 9, 45], or by learning independent transformations [28, 32] to a common latent representation through”
A Broader Study of Cross-Domain Few-Shot Learning (2019, unknown): “More recently, many methods have been proposed to address the problem of domain adaptation using deep neural networks, including discrepancy-based methods, designed to align marginal distributions between the domains [38,54,23,30], adversarial-based approaches, which rely on a domain discriminator t”
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, background): “Adversarial domain-invariant feature alignment has been applied on different levels, following an ensemble scheme (Kumar et al., 2018).”
Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis (2019, methodology): “In the methodology of UDA, one recent work [46] proposes a co-regularization scheme to improve the align-”
HGNet: Hybrid Generative Network for Zero-Shot Domain Adaptation (2020, background): “Although various domains share the high-level semantic information, their data distributions contain signiﬁcant discrepancy deﬁned as domain shift [10, 34, 13].”
A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “[68] outperforms all other methods on the datasets in Table 2, and Co-DA [124] comes close using an ensemble (of size two) of adversarial domain-invariant feature networks.”
Undoing the Damage of Label Shift for Cross-domain Semantic Segmentation (2022, background): “Recently, researchers have found that this alignment strategy may lead to the phenomenon of misalignment [7, 24].”
An Overview of Deep Semi-Supervised Learning (2020, methodology): “As for unsupervised domain adaptation, semi-supervised methods, such as consistency regularization [140, 93, 45], co-regularization [89] or proxy labeling [132, 131] demonstrated their effectiveness in domain adaptation.”
FDA: Fourier Domain Adaptation for Semantic Segmentation (2020, methodology): “However, instead of training multiple models using the same loss at once, with an explicit divergence term as in [21], we directly train multiple models φβ with different β s in the FDA process, with no need to explicitly force model divergence.”
A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation (2020, background): “tion from other incorrect classes. For example, the source output [0.6, 0.3, 0.1] is more uncertain than [0.6, 0.2, 0.2], however, both have the same cross-entropy loss. Even several previous methods [20,34] incorporate virtual adversarial training [31] in the classication term to implicitly increas”
Transfer Learning with Dynamic Adversarial Adaptation Network (2019, background): “Co-DA [37] constructs multiple diverse feature spaces and aligns source and target distributions in each of them individually.”
Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation (2020, background): “However, as discussed in recent works [3,17], matching the marginal feature distributions does not guarantee small expected error on the target domain.”
Efficient image dataset classification difficulty estimation for predicting deep-learning accuracy (2018, background): “TheH -divergence is a rigorousmeasure computedbetween two datasets in the field of domain adaption [2,15,28] where a source and target distribution of datasets are compared.”
Phase Consistent Ecological Domain Adaptation (2020, methodology): “Recent methods apply adversarial learning [13, 39] for classification, by instantiating a discriminator that encourages the alignment in feature space [33, 23, 35].”
Deep Subdomain Adaptation Network for Image Classification (2020, background): “overview of the results, we further compare our DSAN with several adversarial subdomain adaptation methods [14], [15], [18], [20] in Table VI and find some insightful observations.”