Multi-Source Domain Adaptation: A Causal View

By Kun Zhang, Mingming Gong, Bernhard Schölkopf. 2015

208 citations recorded by the source corpus.

This paper is concerned with the problem of domain adaptation with multiple sources from a causal point of view. In particular, we use causal models to represent the relationship between the features X and class label Y , and consider possible situations where different modules of the causal model change with the domain. In each situation, we investigate what knowledge is appropriate to transfer and find the optimal target-domain hypothesis. This gives an intuitive interpretation of the assumptions underlying certain previous methods and motivates new ones. We finally focus on the case where Y is the cause for X with changing PY and PX|Y , that is, PY and PX|Y change independently across domains. Under appropriate assumptions, the availability of multiple source domains allows a natural way to reconstruct the conditional distribution on the target domain; we propose to model PX|Y (the process to generate effect X from cause Y ) on the target domain as a linear mixture of those on source domains, and estimate all involved parameters by matching the target-domain feature distribution. Experimental results on both synthetic and real-world data verify our theoretical results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Epidemiology, Representation & Self-Supervised Learning, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Epidemiology

In Representation & Self-Supervised Learning

In Causal Inference & Discovery

Citations in the corpus

Toward Causal Representation Learning (2021, background): “It also led to a range of follow-up work [32], [78], [97], [114], [115], [152], [153], [156], [167], [195], [204], [243], [263], [267], [277], [278], [281], complementing the studies of Bareinboim and Pearl [14], [185], and it inspired a thread of work in the statistics community exploiting invarian”
Multiple-Source Adaptation with Domain Classifiers (2020, background): “beledandunlabeleddataavailablefromthetargetdomain[Mansour et al.,2008,2009a, Hoffmanet al., 2018, Pan and Yang, 2010, Muandet et al., 2013, Xu et al., 2014, Hoffmanet al., 2012, Gong et al., 2013a,b, Zhang et al., 2015, Ganin et al., 2016, Tzeng et al., 2015, Motiian et al., 2017b,a,Wang et al.,2019”
Agnostic Federated Learning (2019, methodology): “Recently, Zhang et al. (2015) studied a causal formulation of this problem for a classiﬁcation scenario, using the same combination rules as Mansour et al. (2009c,a).”
Algorithms and Theory for Multiple-Source Adaptation (2018, background): “Recently, Zhang et al. [2015] studied a causal formulation of this problem and analyzed the same combination rules of Mansour et al. [2008, 2009] for classiﬁcation scenario.”
A theory of independent mechanisms for extrapolation in generative models (2020, background): “Our ultimate goal is more akin to the use of a causal framework to enforce domain adaptation (Zhang et al. 2013; Zhang, Gong, and Sch¨olkopf 2015) and domain shift robustness of leaning algorithms, which has been done by exploiting additional information in the context of classiﬁcation (Heinze-Deml…”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “A parallel thread of work has studied distribution shifts by identifying changes in causal generative factors (Zhang et al., 2015; 2017; Huang et al., 2017), which is linked to a causal view of disentanglement (Suter et al., 2019; Schölkopf, 2019).”
Causality for Machine Learning (2019, background): “s, thus going beyond the applications or methodological advances that machine learning studies usually provided. The line of work provided rather fruitful (Zhang et al., 2013; Weichwald et al., 2014; Zhang et al., 2015; Gong et al., 2016; Huang et al., 2017; Blöbaum et al., 2016; Guo et al., 2018; W”
Conditional variance penalties and domain shift robustness (2017, background): “…advantage that the predictions will be valid even under arbitrarily large interventions on all predictor variables (Haavelmo, 1944; Aldrich, 1989; Pearl, 2009; Sch¨olkopf et al., 2012; Peters et al., 2016; Zhang et al., 2013, 2015; X. Yu, 2017; M. Rojas-Carulla, 2017; Magliacane et al., 2017).”
Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions (2017, methodology): “The methods proposed in [Schölkopf et al., 2012, Zhang et al., 2013, 2015, Gong et al., 2016] all address challenging settings in which conditional independences that follow from the usual Markov and faithfulness assumptions alone do not sufﬁce to solve the problem, but additional assumptions on…”
Group invariance principles for causal generative models (2017, background): “This includes important cases related to transfer leaning such as covariate-shift or changes in the input distribution [Zhang et al., 2013, 2015; Peters et al., 2016; Bareinboim and Pearl, 2016; Rojas-Carulla et al., 2015], making causal generative models highly relevant in machine learning.”
Causal Transfer in Machine Learning (2015, background): “Moreover, Zhang et al. [2015a] argue that the availability of multiple domains is suﬃcient to drop this previous assumption when the distribution of Y k and the conditional X k | Y k change independently.”
An introduction to domain adaptation and transfer learning (2018, background): “lds One can make an even stronger assumption than that of the existence of a mapping from source to target domain: that there exists a manifold of transformations between the source and target domain [83,7,104,217]. A manifold is a curved lower-dimensional subspace embedded in a larger vector space.”
Matching on Balanced Nonlinear Representations for Treatment Effects Estimation (2017, background): “Meanwhile, causal inference has also been incorporated to improve the performance of domain adaptation [46, 45].”
Generalizing Across Domains via Cross-Gradient Training (2018, background): “…has been studied under many different settings: two domains (Ganin et al., 2016; Tzeng et al., 2017) or multiple domains (Mansour et al., 2009; Zhang et al., 2015), with target domain data that is labeled (III, 2007; III et al., 2010; Saenko et al., 2010) or unlabeled (Gopalan et al., 2011;…”
Representation Learning for Treatment Effect Estimation from Observational Data (2018, background): “Estimating the causal effect of an intervention/treatment at individual-level is an important problem that can benefit many domains including health care [12, 1], digital marketing [6, 34, 24], and machine learning [10, 37, 21, 23, 20].”
Zero-shot Domain Adaptation without Domain Semantic Descriptors (2018, methodology): “Some methods assume multiple source domains for adaptation [25, 44].”