Unsupervised Domain Adaptation with Residual Transfer Networks

By Mingsheng Long, Hanjing Zhu, Jianmin Wang, Michael I. Jordan. 2016

1,609 citations recorded by the source corpus.

The recent success of deep neural networks relies on massive amounts of labeled data. For a target task where labeled data is unavailable, domain adaptation can transfer a learner from a different source domain. In this paper, we propose a new approach to domain adaptation in deep networks that can jointly learn adaptive classifiers and transferable features from labeled data in the source domain and unlabeled data in the target domain. We relax a shared-classifier assumption made by previous methods and assume that the source classifier and target classifier differ by a residual function. We enable classifier adaptation by plugging several layers into deep network to explicitly learn the residual function with reference to the target classifier. We fuse features of multiple layers with tensor product and embed them into reproducing kernel Hilbert spaces to match distributions for feature adaptation. The adaptation can be achieved in most feed-forward models by extending them with new residual layers and loss functions, which can be trained efficiently via back-propagation. Empirical evidence shows that the new approach outperforms state of the art methods on standard domain adaptation benchmarks.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Robust Global Representations by Penalizing Local Predictive Power (2019, methodology): “At a high level these methods comprise two subtypes: fine-tuning over target domain (Long et al., 2016; Hoffman et al., 2017; Motiian et al., 2017a; Gebru et al., 2017; Volpi et al., 2018) and coercing domain invariance via adversarial learning (or further extensions) (Ganin et al.”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “In domain adaptation, for instance, the MMD has also been used to reduce the dataset bias and thus improve transferability of the features across different domains (Long et al. 2015; 2016).”
Learning to cluster in order to Transfer across domains and tasks (2017, background): “…of work dealing with domain shift between image datasets by minimizing domain discrepancy (Tzeng et al., 2014; Long et al., 2015b; Ganin et al., 2016; Sun et al., 2016; Long et al., 2016; Sener et al., 2016; Carlucci et al., 2017; Long et al., 2017; Zellinger et al., 2017; Bousmalis et al., 2017).”
Incremental Learning Through Deep Adaptation (2017, neutral): “([21], [22], [23], [24], [25]).”
Moment Matching for Multi-Source Domain Adaptation (2018, methodology): “Single-Source Adaptation To demonstrate the intrinsic difficulty of DomainNet, we evaluate multiple state-ofthe-art algorithms for single-source adaptation: Deep Alignment Network (DAN) [25], Joint Adaptation Network (JAN) [27], Domain Adversarial Neural Network (DANN) [8], Residual Transfer Network”
Efficient Parametrization of Multi-domain Deep Neural Networks (2018, background): “The authors of [19] propose a deep network architecture that can jointly learn adaptive classifiers and transferable features from the source to target domain by modeling source classifier as sum of target classifier and a residual function.”
A Comprehensive Survey on Transfer Learning (2019, background): “[146] constructed residual transfer networks for domain adaptation, which is motivated by deep residual learning.”
A Survey on Deep Transfer Learning (2018, background): “[9], [17], [15], [30], [3], [6], [28] Adversarial-based Use adversarial technology to ﬁnd transferable features that both suitable for two domains.”
Unsupervised Model Personalization While Preserving Privacy and Scalability: An Open Problem (2020, background): “Finally, deep domain adaptation introduces several unsupervised back-propagation based techniques [6, 19], with state-of-the-art introducing an adversarial loss during training [34, 9].”
Deep Hashing Network for Unsupervised Domain Adaptation (2017, methodology): “[34], trained a network to do both feature adaptation and classifier adaptation using residual transfer networks.”
Dual Transfer Learning for Neural Machine Translation with Marginal Distribution Regularization (2018, background): “Different from most transfer learning methods (Raina et al. 2007; Long et al. 2015; 2016), our algorithm leverages the dual structure of machine translation and achieves knowledge transfer through data sampling.”
Class-Incremental Domain Adaptation (2020, background): “A line of work called Unsupervised Domain Adaptation (UDA) [2,3,5,21,25,31,22,51] has emerged that offers an elegant solution to the domain-shift problem.”
Multi-Adversarial Domain Adaptation (2018, background): “…advances have been achieved by embedding domain adaptation modules in the pipeline of deep feature learning to extract domain-invariant representations (Tzeng et al. 2014; Long et al. 2015; Ganin and Lempitsky 2015; Tzeng et al. 2015; Long et al. 2016; Bousmalis et al. 2016; Long et al. 2017).”
Learning to Adapt Structured Output Space for Semantic Segmentation (2018, methodology): “A number of variants have since been proposed with different loss functions [25, 36, 37] or classifiers [26].”
Contrastive Adaptation Network for Unsupervised Domain Adaptation (2019, background): “A common practice for UDA is to minimize the discrepancy between domains to obtain domain-invariant features [10, 4, 25, 22, 24, 36, 21].”
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation (2016, background): “Other work aims to align the features [21, 22] by minimizing the distance between their distributions in the two domains.”
Deep Transfer Learning with Joint Adaptation Networks (2016, background): “, 2015), RevGrad (Ganin & Lempitsky, 2015), and RTN (Long et al., 2016) are directly reported from their published papers.”
No More Discrimination: Cross City Adaptation of Road Scene Segmenters (2017, background): “[19] minimize the mean distance between data domains, and later they incorporate the concept of residual learning [21] for further improvements.”
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation (2018, methodology): “It is done at single [15, 32, 44] or multiple levels [24, 25] using maximum mean discrepancies (MMD) or adversarial training [10, 42].”