Partial Transfer Learning with Selective Adversarial Networks

By Zhangjie Cao, Mingsheng Long, Jianmin Wang, Michael I. Jordan. 2017

474 citations recorded by the source corpus.

Adversarial learning has been successfully embedded into deep networks to learn transferable features, which reduce distribution discrepancy between the source and target domains. Existing domain adversarial networks assume fully shared label space across domains. In the presence of big data, there is strong motivation of transferring both classification and representation models from existing large-scale domains to unknown small-scale domains. This paper introduces partial transfer learning, which relaxes the shared label space assumption to that the target label space is only a subspace of the source label space. Previous methods typically match the whole source domain to the target domain, which are prone to negative transfer for the partial transfer problem. We present Selective Adversarial Network (SAN), which simultaneously circumvents negative transfer by selecting out the outlier source classes and promotes positive transfer by maximally matching the data distributions in the shared label space. Experiments demonstrate that our models exceed state-of-the-art results for partial transfer learning tasks on several benchmark datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment (2019, background): “Among those, Cao et al. (2018a,b) deal with the label distribution shift scenario through a heuristic reweighting scheme.”
A Comprehensive Survey on Transfer Learning (2019, background): “[163] constructs the selective adversarial network for partial transfer learning.”
KinGDOM: Knowledge-Guided DOMain Adaptation for Sentiment Analysis (2020, methodology): “egorized into three approaches: a) instanceselection (Jiang and Zhai, 2007; Chen et al., 2011; Cao et al., 2018), b) self-labeling (He and Zhou, 2011) and c) representation learning (Glorot et al.”
Under review as a conference paper at ICLR 2020 many domain adaptation methods (2019, background): “The domain adaptation on the partially overlapping source and target domains is presented in Zhang et al. (2018a); Cao et al. (2018).”
Digital twin-driven partial domain adaptation network for intelligent fault diagnosis of rolling bearing (2023, background): “To match the simulated and measured data of nonidentical health condition spaces C s (cid:54) = C t , the selective adversarial network is introduced to implement partial domain adaptation [49].”
A Survey on Negative Transfer (2020, background): “[105] proposed a weighted adversarial nets based approach to identify source samples that are potential outliers.”
Domain Consensus Clustering for Universal Domain Adaptation (2021, background): “SAN [3] employs class-wise domain discriminators to align the distributions in a ﬁne-grained manner.”
Applications of Unsupervised Deep Transfer Learning to Intelligent Fault Diagnosis: A Survey and Comparative Study (2019, background): “1) Basic concepts: Partial UDTL, which was proposed in [135], is a transfer learning paradigm where the target label set Ct is a subspace of the source label set Cs, i.”
Universal Source-Free Domain Adaptation (2020, methodology): “In the first kind, Partial DA [54, 5, 6], the target label space is considered as a subset of the source label space (i.”
A Survey of Unsupervised Deep Domain Adaptation (2018, background): “ce gains to other problems as well. Finally, another under-studied case of domain adaptation is partial domain adaptation, where the target domain contains only a subset of the source domain’s labels [26,233,272]. 9 CONCLUSIONS For supervised learning, deep neural networks are in prevalent use, but ”
A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation (2020, methodology): “Inspired by the observation in [4] that redundant information is not beneficial for adaptation, we adopt the following optimization objective of Entropy-regularized DANN (E-DANN) as an initial model of our method,”
Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review (2018, background): “117 However, this is still an open area for deep learning on EHR data.”
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, background): “Due to the mismatch of categories between source and target domains, conventional UDA approaches may result in negative transfer (Cao et al., 2018b,a; Kim et al., 2020b).”
Importance Weighted Adversarial Nets for Partial Domain Adaptation (2018, methodology): “This paper is concerned with a different and more practical scenario that the target domain only has a subset of classes, referred to as partial domain adaptation (similar to [4]).”
Deep Visual Domain Adaptation: A Survey (2018, background): “Unlike previous methods that match the entire source and target domains, Cao et al. introduced a selective adversarial network (SAN) [8] to address partial transfer learning from large domains to small domains, which assumes that the target label space is a subspace of the source label space.”
Overcoming Negative Transfer: A Survey (2020, background): “Then Cao et al. [45] proposed example transfer network (ETN), which down-weights the irrelevant examples of outlier classes further on the source classiﬁer by adding the class-level weight proposed in [44] into classiﬁcation loss function.”