Adversarial Discriminative Domain Adaptation
By Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell. 2017
5,283 citations recorded by the source corpus.
Adversarial learning methods are a promising approach to training robust deep networks, and can generate complex samples across diverse domains. They can also improve recognition despite the presence of domain shift or dataset bias: recent adversarial approaches to unsupervised domain adaptation reduce the difference between the training and test domain distributions and thus improve generalization performance. However, while generative adversarial networks (GANs) show compelling visualizations, they are not optimal on discriminative tasks and can be limited to smaller shifts. On the other hand, discriminative approaches can handle larger domain shifts, but impose tied weights on the model and do not exploit a GAN-based loss. In this work, we first outline a novel generalized framework for adversarial adaptation, which subsumes recent state-of-the-art approaches as special cases, and use this generalized view to better relate prior approaches. We then propose a previously unexplored instance of our general framework which combines discriminative modeling, untied weight sharing, and a GAN loss, which we call Adversarial Discriminative Domain Adaptation (ADDA). We show that ADDA is more effective yet considerably simpler than competing domain-adversarial methods, and demonstrate the promise of our approach by exceeding state-of-the-art unsupervised adaptation results on standard domain adaptation tasks as well as a difficult cross-modality object classification task.
Subfields: Artificial Intelligence, Ocean Engineering, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dacheng Tao
- Yoshua Bengio
- Sergey Levine
- Mohit Bansal
- Trevor Darrell
- Jianfeng Gao
- Percy Liang
- Quoc V. Le
In Ocean Engineering
- Yi Yang
- Stefano Soatto
- Mingsheng Long
- Tao A. Xiang
- Michael I. Jordan
- Jianmin Wang
- Alan L. Yuille
- Andrew Zisserman
In Representation & Self-Supervised Learning
- Dacheng Tao
- Trevor Darrell
- Mohammad Rostami
- Mingsheng Long
- Jianmin Wang
- Kate Saenko
- Luc Van Gool
- Jiashi Feng
Citations in the corpus
- Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “…Volpi et al., 2018) and coercing domain invariance via adversarial learning (or further extensions) (Ganin et al., 2016; Bousmalis et al., 2017; Tzeng et al., 2017; Xie et al., 2018; Hoffman et al., 2018; Long et al., 2018; Zhao et al., 2018b; Kumar et al., 2018; Li et al., 2018b; Zhao et…”
- VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “…al., 2020) and speech (Ping et al., 2020; Oord et al., 2016a) generation, distribution alignment across domains (Zhu et al., 2017; Liu et al., 2017; Tzeng et al., 2017) and semi-supervised learning (Kingma et al., 2014; Izmailov et al., 2020) to 3D point cloud generation (Yang et al., 2019),…”
- DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-To-Image Synthesis (2019, background): “With the recent successes of Variational Autoencoders (VAEs) [9] and GANs [4], a large number of methods have been proposed to handle generation [14, 17, 28, 1] and domain adaptation task [25, 32].”
- Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning (2021, background): “GAN-based DA approaches require having access to the large amount of unlabeled data from both the source and target datasets (Tzeng et al., 2017; Zhang et al., 2019; Wilson & Cook, 2020).”
- The Numerics of GANs (2017, background): “Since their first appearance, GANs have been successfully applied to a variety of tasks, including image-to-image translation [12], image super-resolution [13], image in-painting [27] domain adaptation [26], probabilistic inference [14, 9, 8] and many more.”
- Toward Domain-Invariant Speech Recognition via Large Scale Training (2018, background): “Domain adversarial training and variants are widely used for this [18, 19].”
- Domain Adaptation for Traffic Density Estimation (2021, background): “Traditional UDA approaches have been developed to address the problem of image classification, and they try to align features across the two domains ((Ganin and Lempitsky, 2015), (Tzeng et al., 2017)).”
- FRAGE: Frequency-Agnostic Word Representation (2018, background): “Our work is also weakly related to adversarial domain adaptation which attempts to mitigate the negative effects of domain shift between training and testing [9, 36].”
- Learning Independent Causal Mechanisms (2017, methodology): “These methods often use some supervision from labeled data and/or match the two distributions in a learned feature space (e.g. Tzeng et al., 2017).”
- Few-Shot Adversarial Domain Adaptation (2017, methodology): “The first two tasks includeM→ U , U →M, and followed the experimental setting in [12, 45, 33, 59, 49], which involves randomly selecting 2000 images from MNIST and 1800 images from USPS.”
- Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptation (2022, methodology): “C→I Step 1 [0, 2, 7, 9, 11, 27, 28, 29, 30, 33] Step 2 [37, 39, 40, 44, 45, 47, 50, 60, 62, 68] Step 3 [71, 75, 76, 82, 85, 86, 87, 88, 89, 90] Step 4 [92, 94, 96, 97, 106, 107, 108, 109, 110, 112] Step 5 [114, 115, 116, 123, 126, 128, 133, 134, 141, 145] Step 6 [146, 150, 151, 157, 160, 163, 165, 1”
- Adversarial Continual Learning (2020, methodology): “Adversarial learning has been used for different problems such as generative models [13], object composition [2], representation learning [22], domain adaptation [36], active learning [34], etc.”
- Incremental Learning Through Deep Adaptation (2017, neutral): “([21], [22], [23], [24], [25]).”
- Moment Matching for Multi-Source Domain Adaptation (2018, methodology): “The second category is the adversarial-based approach [24, 40].”
- CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017, methodology): “Even a slight departure from a network’s training domain can cause it to make spurious predictions and significantly hurt its performance (Tzeng et al., 2017).”
- A Comprehensive Survey on Transfer Learning (2019, background): “[156] proposes a unified adversarial domain adaptation framework.”