Joint distribution optimal transportation for domain adaptation

By Nicolas Courty, Rémi Flamary, Amaury Habrard, Alain Rakotomamonjy. 2017

681 citations recorded by the source corpus.

This paper deals with the unsupervised domain adaptation problem, where one wants to estimate a prediction function $f$ in a given target domain without any labeled sample by exploiting the knowledge available from a source domain where labels are known. Our work makes the following assumption: there exists a non-linear transformation between the joint feature/label space distributions of the two domain $\mathcal{P}_s$ and $\mathcal{P}_t$. We propose a solution of this problem with optimal transport, that allows to recover an estimated target $\mathcal{P}^f_t=(X,f(X))$ by optimizing simultaneously the optimal coupling and $f$. We show that our method corresponds to the minimization of a bound on the target error, and provide an efficient algorithmic solution, for which convergence is proved. The versatility of our approach, both in terms of class of hypothesis or loss functions is demonstrated with real world classification and regression problems, for which we reach or surpass state-of-the-art results.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “Wasserstein Distance (WD) has been used successfully for domain alignment in the UDA literature (Courty, Flamary, and Tuia 2014; Courty et al. 2017; Bhushan Damodaran et al. 2018b; Xu et al. 2020; Li et al. 2020).”
DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation (2018, methodology): “Despite similarities with the formulation of JDOT [5], our proposition comes with the notable difference that, in DeepJDOT, the Wasserstein distance is minimized between the joint (embedded space/label) distributions within the CNN, rather than between the original input spaces.”
Bridging Theory and Algorithm for Domain Adaptation (2019, methodology): “Furthermore, there is another gap between the hypothesis-induced discrepancies in theories and the widely-used divergences in domain adaptation algorithms, including Jensen Shannon Divergence (Ganin & Lempitsky, 2015), Maximum Mean Discrepancy (Gretton et al., 2012; Long et al., 2015), and Wasserste”
Conditional Adversarial Domain Adaptation (2017, background): “Domain adaptation [38, 39] generalizes a learner across different domains of different distributions, by either matching the marginal distributions [49, 37, 15] or the conditional distributions [55, 10].”
Small Sample Learning in Big Data Era (2018, background): “Courty et al. (2017a) went a step further to implicitly learn a non-linear transformation that minimized the optimal transport loss between the joint source distribution and an estimated target joint distribution, corresponding to the minimization of a bound on the target error.”
A Review of Domain Adaptation without Target Labels (2019, unknown): “but could also be joint distrbutions [135].”
Rectify Heterogeneous Models with Semantic Mapping (2018, methodology): “For example, in image query (Rubner et al., 1998), document classification (Huang et al., 2016), domain adaptation (Perrot et al., 2016; Courty et al., 2017a;b), and barycenter discovery (Cuturi & Doucet, 2014).”
Support and Invertibility in Domain-Invariant Representations (2019, background): “, 2013) and the Wasserstein distance (Shalit et al., 2016; Courty et al., 2017), or other divergences (Berisha et al.”
A Survey of Unsupervised Deep Domain Adaptation (2018, background): “e incorporated. While domain adaptation methods commonly align feature distributions, a different line of research aligns the joint or conditional distribution of the feature and label spaces instead [44, 47, 50, 148, 151, 154, 233, 267]. Researchers found aligning in this manner improves results wh”
Deep Domain Generalization via Conditional Invariant Adversarial Networks (2018, result): “[26] proposed an optimal transport-based approach to matching joint distributions and obtained promising results.”
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach (2018, methodology): “Recently proposed unsupervised DA methods ( [ 8 , 14 , 33 , 36 , 37 , 46 ]) operate by training deep neural networks using adversarial training, which allows the learning of feature representations that are simultaneously discriminative of source labels, and indistinguishable between the source and…”
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, methodology): “The joint distribution optimal transport (JDOT) is proposed to measure the Wasserstein distance between two feature distributions (Courty et al., 2017).”
Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation (2019, methodology): “JDOT method [9] learns to map input space from source to target by jointly considering the class regularity and feature distribution.”
Adaptation Across Extreme Variations using Unlabeled Domain Bridges (2019, methodology): “Recent works use kernel-based methods such as maximum mean discrepancy (MMD) [34, 52, 54, 35] and optimum transport (OT) [10, 11] to measure the domain difference in the feature level space.”
On Learning Invariant Representations for Domain Adaptation (2019, methodology): “…on using the Hdivergence to characterize the discrepancy between two distributions, other distance measures can be used as well, e.g., the maximum mean discrepancy (MMD) (Long et al., 2014; 2015; 2016), the Wasserstein distance (Courty et al., 2017b;a; Shen et al., 2018; Lee & Raginsky, 2018), etc.”
Domain Adaptation by Joint Distribution Invariant Projections (2020, background): “Although matching the marginal distributions in this way has been justified theoretically [33], as mentioned in [4], [34], [35], this does not imply that the labeling distributions are matched as well.”