Deep Transfer Learning with Joint Adaptation Networks
By Mingsheng Long, Hanhua Zhu, Jianmin Wang, Michael I. Jordan. 2016
2,790 citations recorded by the source corpus.
Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain. In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy (JMMD) criterion. Adversarial training strategy is adopted to maximize JMMD such that the distributions of the source and target domains are made more distinguishable. Learning can be performed by stochastic gradient descent with the gradients computed by back-propagation in linear-time. Experiments testify that our model yields state of the art results on standard datasets.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dacheng Tao
- Jiashi Feng
- Trevor Darrell
- Luc Van Gool
- Kate Saenko
- Yoshua Bengio
- Sergey Levine
- Mohit Bansal
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Yi Yang
- Xiaogang Wang
- Jiashi Feng
- Jitendra Malik
In Representation & Self-Supervised Learning
- Kate Saenko
- Dacheng Tao
- Mingsheng Long
- Gabriella Csurka
- Jianmin Wang
- Mohammad Rostami
- Trevor Darrell
- Jiashi Feng
In Meta & Continual Learning
- Dacheng Tao
- Jun Zhu
- Mohammad Rostami
- Simone Calderara
- Matteo Boschini
- Lorenzo Bonicelli
- Mingming Gong
- Kate Saenko
Citations in the corpus
- Domain Generalization via Conditional Invariant Representations (2018, methodology): “Domain adaptation is a related problem which has been extensively studied in the literature (Baktashmotlagh et al. 2013; Huang et al. 2007; Pan et al. 2011; Long et al. 2017; Shao, Kit, and Fu 2014; Shao et al. 2016; Luo et al. 2017; Liu, Yang, and Tao 2017).”
- Learning to cluster in order to Transfer across domains and tasks (2017, methodology): “Following the standard evaluation procedure (Long et al., 2017; Ganin et al., 2016), the labeled datasets A is ImageNet and S (cid:48) is one domain in the Office-31 dataset.”
- Transfer without Forgetting (2022, methodology): “Such a simple schema has been recently overcome by more sophisticated domain adaptation algorithms [15,36,35] mainly based on the concept of feature alignment : here, the goal is to reduce the shift between the feature distributions of target and source domains.”
- Moment Matching for Multi-Source Domain Adaptation (2018, methodology): “Inspired by the kernel two-sample test [12], Maximum Mean Discrepancy (MMD) is applied to reduce distribution shift in various methods [27, 41, 9, 44].”
- A Comprehensive Survey on Transfer Learning (2019, methodology): “, the Alexnet [138], [140] pretrained on ImageNet [166] and then directly trained on the target domain) are performed on the data set Office-31 (here, we directly use the results of CDAN, JAN, CAN, and the baseline from the original articles [137], [147], [151], [158]).”
- A Survey on Deep Transfer Learning (2018, background): “[23], [12], [8], [14], [2] Network-based Reuse the partial of network pre-trained in the source domain.”
- Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, methodology): “mposition [126] on intermediate datasets, where the amount of source data is gradually replaced by target samples. Domain Adaptation for Visual Applications: A Comprehensive Survey 23 Fig. 19 The JAN [142] minimizes a joint distribution discrepancy of several intermediate layers including the soft p”
- Multi-Adversarial Domain Adaptation (2018, background): “…advances have been achieved by embedding domain adaptation modules in the pipeline of deep feature learning to extract domain-invariant representations (Tzeng et al. 2014; Long et al. 2015; Ganin and Lempitsky 2015; Tzeng et al. 2015; Long et al. 2016; Bousmalis et al. 2016; Long et al. 2017).”
- Style Transfer in Text: Exploration and Evaluation (2017, background): “(Long, Wang, and Jordan 2017) proposed a joint adaptation network, which adopted the adversarial strategy to maximize joint maximum mean discrepancy.”
- Contrastive Adaptation Network for Unsupervised Domain Adaptation (2019, methodology): “[21].”
- Scalable Transfer Learning with Expert Models (2020, background): “[36] M.”
- Adversarial Domain Adaptation With Prototype-Based Normalized Output Conditioner (2020, methodology): “[34] proposes to align joint distributions of multiple domain-specific layers across domains based on a joint MMD criterion.”
- A Survey of Deep Learning for Scientific Discovery (2020, background): “Some popular ways to implement this include gradient reversal [62], minimizing a distance function on the hidden representations [137], and even adversarial training [63, 211].”
- Progressive Domain Adaptation for Object Detection (2019, methodology): “Numerous methods are developed in the context of image classification [32, 18, 19, 28, 10, 31, 7, 2], while fewer efforts have been made on more complicated tasks such as semantic segmentation [13, 29] and object detection [11, 3, 15].”
- Denoised Maximum Classifier Discrepancy for Source-Free Unsupervised Domain Adaptation (2022, background): “…between the source and target domains in the feature space and rely on matching the high-order moments of the source domain and the target domain (Tzeng et al. 2014; Long et al. 2017; Gretton et al. 2012) discriminator to learn domain-invariant features (Ganin and Lempitsky 2015; Long et al. 2018).”
- Partial Adversarial Domain Adaptation (2018, methodology): “Along this line, domain adaptation modules such as moment matching [6,7,8,9] and adversarial adaptation [10,11,12] have been embedded in deep networks to learn domain-transferable representations.”