Transferable Representation Learning with Deep Adaptation Networks

By Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I. Jordan. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

643 citations recorded by the source corpus.

Domain adaptation studies learning algorithms that generalize across source domains and target domains that exhibit different distributions. Recent studies reveal that deep neural networks can learn transferable features that generalize well to similar novel tasks. However, as deep features eventually transition from general to specific along the network, feature transferability drops significantly in higher task-specific layers with increasing domain discrepancy. To formally reduce the effects of this discrepancy and enhance feature transferability in task-specific layers, we develop a novel framework for deep adaptation networks that extends deep convolutional neural networks to domain adaptation problems. The framework embeds the deep features of all task-specific layers into reproducing kernel Hilbert spaces (RKHSs) and optimally matches different domain distributions. The deep features are made more transferable by exploiting low-density separation of target-unlabeled data in very deep architectures, while the domain discrepancy is further reduced via the use of multiple kernel learning that enhances the statistical power of kernel embedding matching. The overall framework is cast in a minimax game setting. Extensive empirical evidence shows that the proposed networks yield state-of-the-art results on standard visual domain-adaptation benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/83d410601e3d55fab285926882828a48e23b97e4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jian Liang
2. Jiashi Feng
3. Dapeng Hu
4. Zhangjie Cao
5. Ran He
6. Mingsheng Long
7. Jianmin Wang
8. Yunbo Wang

In Computer Vision and Pattern Recognition

1. Jian Liang
2. Jiashi Feng
3. Dapeng Hu
4. Ran He
5. Mingsheng Long
6. Jianmin Wang
7. Zhangjie Cao
8. Luc Van Gool

Citations in the corpus

• A Decade Survey of Transfer Learning (2010–2020) (2020, methodology): “ToexplorethepotentialofDaNN,a numberofnovelmethodswereproposed[16],[18]–[21].”
• Zoo-Tuning: Adaptive Transfer from a Zoo of Models (2021, background): “Long et al. (2015; 2018) further proposed to learn transferable representations.”
• A Comprehensive Survey on Transfer Learning (2019, background): “[141] further extended the above DAN approach and proposed the DAN framework.”
• Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer (2020, methodology): “been exploited in many previous UDA methods [41], [82].”