Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation

By Hongliang Yan, Yukang Ding, P. Li, Qilong Wang, Yong Xu, Wangmeng Zuo. Computer Vision and Pattern Recognition, 2017

650 citations recorded by the source corpus.

In domain adaptation, maximum mean discrepancy (MMD) has been widely adopted as a discrepancy metric between the distributions of source and target domains. However, existing MMD-based domain adaptation methods generally ignore the changes of class prior distributions, i.e., class weight bias across domains. This remains an open problem but ubiquitous for domain adaptation, which can be caused by changes in sample selection criteria and application scenarios. We show that MMD cannot account for class weight bias and results in degraded domain adaptation performance. To address this issue, a weighted MMD model is proposed in this paper. Specifically, we introduce class-specific auxiliary weights into the original MMD for exploiting the class prior probability on source and target domains, whose challenge lies in the fact that the class label in target domain is unavailable. To account for it, our proposed weighted MMD model is defined by introducing an auxiliary weight for each class in the source domain, and a classification EM algorithm is suggested by alternating between assigning the pseudo-labels, estimating auxiliary weights and updating model parameters. Extensive experiments demonstrate the superiority of our weighted MMD over conventional MMD for domain adaptation.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0d9decf6e3ce9c4f2239b9f0731ab9107ade9b7e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

Citations in the corpus

A Decade Survey of Transfer Learning (2010–2020) (2020, methodology): “Generally, there are two types of feature-based transductive learning methods: data-centered methods [16]–[19], [23] and subspace-centered methods [21], [22], [24]–[26].”
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach (2018, background): “…on transferring deep neural network representations from a labeled source dataset to an unlabeled target domain, where the main strategy is to ﬁnd a feature space such that the confusion between source and target distributions in that space is maximized ( [ 8 , 10 , 14 , 33 , 36 , 37 , 43 , 46 ]).”
Balanced Distribution Adaptation for Transfer Learning (2017, methodology): “Classification accuracy on target domain is adopted as the evaluation metric which is widely used in literatures [2], [3].”
Transfer Learning with Dynamic Adversarial Adaptation Network (2019, methodology): “For instance, Correlation Alignment (CORAL) [17], Kullback-Leibler (KL) divergence [31], Maximum Mean Discrepancy (MMD) [32], [21], [33], [34], [15], and Central Moment Discrepancy (CMD) [35] are used to reduce the distribution divergence between domains.”
Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift (2018, methodology): “Inspired by the two-sample test [17], domain discrepancy based methods, e.g., shallow-model-based TCA [37], JDA [1]; deep-model-based DAN [29], WMMD [48], RTN [30], leverage different distribution measures as domain regular-izer to attain domain-invariant feature.”
Small Sample Learning in Big Data Era (2018, methodology): “The early works proposed in this manner were constructed based on instance re-weighting (Zadrozny, 2004; Sugiyama et al., 2008; Kanamori et al., 2009; Huang et al., 2007; Yan et al., 2017), which estimated the ratio between the likelihoods of being a source or target example or use maximum mean disc”
Towards Backward-Compatible Representation Learning (2020, methodology): “In domain adaptation [20, 4, 36], techniques such as MMD [41] and related methods [32, 18, 41], can be used to align the (marginal) distribution of the new and old classes, included those trained adversarially [11].”
Generalizing a Person Retrieval Model Hetero- and Homogeneously (2018, methodology): “Most of the previous methods try to align the source to the target domain by reducing the divergence of feature distributions [3,4,8,9,10,11].”
Progressive Feature Alignment for Unsupervised Domain Adaptation (2018, methodology): “Many approaches utilize a distance metric to measure the domain discrepancy between the source and target domains, such as maximum mean discrepancy (MMD), KLdivergence or Wasserstein distance [12, 22, 37, 24, 42, 6].”
Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (2020, methodology): “There also exist methods (Tzeng et al., 2014; Long et al., 2015; Yan et al., 2017; Saito et al., 2018; Ghifary et al., 2016) that minimize the Maximum Mean Discrepancy (MMD) loss between source- and target-domain distributions.”
Deep Visual Domain Adaptation: A Survey (2018, methodology): “[34] proposed a eighted MMD model that introduces an auxiliary weight for each lass in the source domain when the class weights in the target omain are not the same as those in the source domain.”
Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-Identification (2019, methodology): “This alignment can be achieved by reducing the Maximum Mean Discrepancy (MMD) [11] between domains [17, 35], or training an adversarial domain-classifier [2, 27] to encourage the features of the source and target domains to be indistinguishable.”
Open Set Domain Adaptation by Backpropagation (2018, methodology): “The distance is utilized to train domain-invariantly eﬀective neural networks, and its variants are proposed [6,7,22,23].”
Unsupervised Person Re-Identification via Multi-Label Classification (2020, background): “For example, as discussed in many works [21, 35, 31], the performance of transfer learning is closely related to the domain gap, e.”
Heterogeneous Graph Attention Network for Unsupervised Multiple-Target Domain Adaptation (2020, background): “Moreover, the work [34] explores a weighted Maximum Mean Discrep-”
Adversarial Dropout Regularization (2017, methodology): “Recent unsupervised domain adaptation (UDA) methods for visual data aim to align the feature distributions of the source and target domains (Sun et al. (2016); Sun & Saenko (2016); Tzeng et al. (2014); Ganin et al. (2016); Long et al. (2015b); Yan et al. (2017); Long et al. (2017)).”
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation (2018, background): “It is done at single [15, 32, 44] or multiple levels [24, 25] using maximum mean discrepancies (MMD) or adversarial training [10, 42].”
Trainable Calibration Measures For Neural Networks From Kernel Mean Embeddings (2018, methodology): “, 2015) and unsupervised domain-adaptation models in (Yan et al., 2017).”