Improve Unsupervised Domain Adaptation with Mixup Training

By Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, Liu Ren. 2020

275 citations recorded by the source corpus.

Unsupervised domain adaptation studies the problem of utilizing a relevant source domain with abundant labels to build predictive modeling for an unannotated target domain. Recent work observe that the popular adversarial approach of learning domain-invariant features is insufficient to achieve desirable target domain performance and thus introduce additional training constraints, e.g. cluster assumption. However, these approaches impose the constraints on source and target domains individually, ignoring the important interplay between them. In this work, we propose to enforce training constraints across domains using mixup formulation to directly address the generalization performance for target data. In order to tackle potentially huge domain discrepancy, we further propose a feature-level consistency regularizer to facilitate the inter-domain constraint. When adding intra-domain mixup and domain adversarial learning, our general framework significantly improves state-of-the-art performance on several important tasks from both image classification and human activity recognition.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Pediatrics, Perinatology and Child Health

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Divyat Mahajan
2. Seunghyun Park
3. Amit Sharma
4. Shruti Tople
5. Enzo Ferrante
6. Diego H. Milone
7. Lucas Mansilla
8. Rodrigo Echeveste

In Computer Vision and Pattern Recognition

1. Seunghyun Park
2. Jinkyu Kim
3. Enzo Ferrante
4. Shruti Tople
5. Amit Sharma
6. Divyat Mahajan
7. David Lopez-Paz
8. Diego H. Milone

In Pediatrics, Perinatology and Child Health

1. Enzo Ferrante

Citations in the corpus

• In Search of Lost Domain Generalization (2020, background): “• Inter-domain Mixup ( Mixup , Xu et al. [2019], Yan et al. [2020], Wang et al. [2020]) performs ERM on linear interpolations of examples from random pairs of domains and their labels.”
• Domain Generalization using Causal Matching (2020, methodology): “(Carlucci et al., 2019), S-MLDG (Li et al., 2020), D-SAM (D’Innocente & Caputo, 2018), MMLD (Matsuura & Harada, 2020), DDAIG (Zhou et al., 2020) SagNet (Nam et al., 2019), DDEC (Asadi et al., 2019), DANN (Ganin et al., 2016), C-DANN (Li et al., 2018d), DRO (Sagawa et al., 2019), Mixup (Xu et al., 20”
• SelfReg: Self-supervised Contrastive Regularization for Domain Generalization (2021, methodology): “This testbed currently provides seven multi-domain datasets (i.e. ColoredMNIST [1], RotatedMNIST [16], VLCS [13], PACS [26], Office-Home [42], and TerraIncognita [2], DomainNet [34]) and provides benchmarks results of 14 baseline approaches (i.e. ERM [41], IRM [1], GroupDRO [36], Mixup [45], MLDG [27”
• Domain Generalization via Gradient Surgery (2021, methodology): “We also included four DG state-of-the-art (SOTA) methods in the comparison: Invariant Risk Minimization (IRM) [1], Meta-Learning Domain Generalization (MLDG) [14], Inter-domain Mixup (Mixup) [34] and Group Distributionally Robust Optimization (DRO) [23].”
• Gradient Matching for Domain Generalization (2021, unknown): “…domains (Sun and Saenko, 2016), learning useful representations by solving Jigsaw puzzles (Carlucci et al., 2019), using the maximum mean discrepancy to match the feature distributions (Li et al., 2018b) or introducing training constraints across domains using mixup formulation (Yan et al., 2020).”