Balanced Distribution Adaptation for Transfer Learning

By Jindong Wang, Yiqiang Chen, Shuji Hao, Wenjie Feng, Zhiqi Shen. Industrial Conference on Data Mining, 2017

540 citations recorded by the source corpus.

Transfer learning has achieved promising results by leveraging knowledge from the source domain to annotate the target domain which has few or none labels. Existing methods often seek to minimize the distribution divergence between domains, such as the marginal distribution, the conditional distribution or both. However, these two distances are often treated equally in existing algorithms, which will result in poor performance in real applications. Moreover, existing methods usually assume that the dataset is balanced, which also limits their performances on imbalanced tasks that are quite common in real problems. To tackle the distribution adaptation problem, in this paper, we propose a novel transfer learning approach, named as Balanced Distribution Adaptation (BDA), which can adaptively leverage the importance of the marginal and conditional distribution discrepancies, and several existing methods can be treated as special cases of BDA. Based on BDA, we also propose a novel Weighted Balanced Distribution Adaptation (W-BDA) algorithm to tackle the class imbalance issue in transfer learning. W-BDA not only considers the distribution adaptation between domains but also adaptively changes the weight of each class. To evaluate the proposed methods, we conduct extensive experiments on several transfer learning tasks, which demonstrate the effectiveness of our proposed algorithms over several state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8334a561ff546c94210c3951a23215416913345e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Compatible Embeddings (2021, unknown): “JDA is further extended by utilizing the label and structure information [18], clustering information [41], various statistical and geometrical information [53] and balanced distributions [47], etc.”
Applications of Unsupervised Deep Transfer Learning to Intelligent Fault Diagnosis: A Survey and Comparative Study (2019, background): “There are many methods belonging to mapping-based UDTL, such as Euclidean distance, Minkowski distance, Kullback-Leibler, correlation alignment (CORAL) [59], maximum mean discrepancy (MMD) [60], [61], multi kernels MMD (MK-MMD) [21], [62], joint distribution adaptation (JDA) [63], balanced distribut”
A Review of Domain Adaptation without Target Labels (2019, methodology): “Balanced Distribution Adaptation (BDA) uses a source classifier to obtain soft labels for the target samples [189].”
Deep Subdomain Adaptation Network for Image Classification (2020, background): “CMMD [7], [23], [33] is a special case of”
A Comprehensive Survey on Transfer Learning (2019, methodology): “where μ is a parameter balancing the marginal and the conditional distribution difference [70], ΧGEO(Φ) is a regularizer controlling the geometric structure, Φ(X) is the matrix whose rows are the instances from both the source and the target domains with the extracted new feature representation, H i”
Visual Domain Adaptation with Manifold Embedded Distribution Alignment (2018, methodology): “ with CORAL loss Deep Unsupervised Convolutional Domain Adaptation (DUCDA) [41], which is based on a−ention and CORAL loss 4.3 Experimental Setup For fair comparison, we follow the same protocols as [37, 40, 41] to adopt the extracted features for MEDA and other traditional methods. To be speci•c, 2”
Deep Learning for Sensor-based Activity Recognition: A Survey (2017, background): “Transfer learning performs data annotation by leveraging labeled data from other auxiliary domains (Pan and Yang, 2010; Cook et al., 2013; Wang et al., 2017).”
Domain Adaptation for Medical Image Analysis: A Survey (2021, unknown): “…they evaluate three shallow domain adaptation methods, i.e. , Transfer Component Analysis (TCA) [67], Correlation Alignment (CORAL) [68], and Balanced Distribution Adaptation (BDA) [69] for breast cancer classiﬁcation, and provide some empirical results on the DDSM [70] and InBreast [71] datasets.”
Transfer Learning with Dynamic Adversarial Adaptation Network (2019, methodology): “Two more recent work called Balanced Distribution Adaptation (BDA) [8] and Manifold Embedded Distribution Alignment (MEDA) [7] proposed to adaptively align these two distributions, while it is based on kernel method with high computational cost.”
LEEP: A New Measure to Evaluate Transferability of Learned Representations (2020, background): “Imbalanced target data sets are commonly encountered in practice (Al-Stouhi & Reddy, 2016; Wang et al., 2017).”
Stratified Transfer Learning for Cross-domain Activity Recognition (2017, background): “fruitful line of work has been done in this area [30], [31], [10].”
Reliable Weighted Optimal Transport for Unsupervised Domain Adaptation (2020, background): “Wang et al. [44, 42, 41] dynamic evaluate the importance of marginal and conditional distributions for distribution alignment.”