Deep Transfer Network: Unsupervised Domain Adaptation

By Xu Zhang, Felix X. Yu, Shih-Fu Chang, Shengjin Wang. 2015

143 citations recorded by the source corpus.

Domain adaptation aims at training a classifier in one dataset and applying it to a related but not identical dataset. One successfully used framework of domain adaptation is to learn a transformation to match both the distribution of the features (marginal distribution), and the distribution of the labels given features (conditional distribution). In this paper, we propose a new domain adaptation framework named Deep Transfer Network (DTN), where the highly flexible deep neural networks are used to implement such a distribution matching process. This is achieved by two types of layers in DTN: the shared feature extraction layers which learn a shared feature subspace in which the marginal distributions of the source and the target samples are drawn close, and the discrimination layers which match conditional distributions by classifier transduction. We also show that DTN has a computation complexity linear to the number of training samples, making it suitable to large-scale problems. By combining the best paradigms in both worlds (deep neural networks in recognition, and matching marginal and conditional distributions in domain adaptation), we demonstrate by extensive experiments that DTN improves significantly over former methods in both execution time and classification accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Women also Snowboard: Overcoming Bias in Captioning Models (2018, neutral): “Similar techniques which incorporate distribution matching have been used for other problems including domain adaptation [18,19] and transduction learning [20].”
Deep Visual Domain Adaptation: A Survey (2018, background): “[33] proposed DTN, where both the marginal and he conditional distributions are matched based on MMD (Fig.”
Label Efficient Learning of Transferable Representations acrosss Domains and Tasks (2017, methodology): “Several methods have tackled the problem with the Maximum Mean Discrepancy (MMD) loss [17, 36, 37, 38, 73] between the source and target domain.”
A Brief Review of Domain Adaptation (2020, background): “Deep Transfer Network (DTN) [58], was proposed to match both marginal and conditional distributions simultaneously.”
Weakly-Shared Deep Transfer Networks for Heterogeneous-Domain Knowledge Propagation (2015, background): “Zhang et al. [36] proposed a deep neural network for domain adaptation by modeling and matching both the marginal and the conditional distribution between two homogeneous data.”
Deep Transfer Learning for Person Re-Identification (2016, methodology): “Finally, our models achieve a good performance of deep Re-ID models: Rank-1 accuracy of 85.4%, 83.7% and 56.3% on CUHK03, Market1501, and VIPeR respectively; and our unsupervised model rank-1 accuracy 45.1% on VIPeR.”
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach (2018, methodology): “We have similar observations for ADDA , DTN , and DSN that demonstrates a naive combination of different target datasets can sometimes even decrease the performance of the competing methods.”
Subspace Alignment Based Domain Adaptation for RCNN Detector (2015, methodology): “In very recent work a more sophisticated technique has been proposed by Zhang et al [14] where a deep transfer network is learned that learns a shared feature subspace that matches conditional distributions.”
Reliable Weighted Optimal Transport for Unsupervised Domain Adaptation (2020, methodology): “Deep Transfer Network (DTN) [49] utilizes pseudo label in conditional alignment.”
Recycling Privileged Learning and Distribution Matching for Fairness (2017, methodology): “[34], where they use the MMD metric to match both the distribution of the features and the distribution of the labels given features in the source and target domains.”
A Digital-Twin-Assisted Fault Diagnosis Using Deep Transfer Learning (2019, background): “It extracts knowledge from one or more source domains for using in a related, but different target domain [29], [30].”