Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions

By Sara Magliacane, Thijs van Ommen, Tom Claassen, Stephan Bongers, Philip Versteeg, Joris M. Mooij. 2017

278 citations recorded by the source corpus.

An important goal common to domain adaptation and causal inference is to make accurate predictions when the distributions for the source (or training) domain(s) and target (or test) domain(s) differ. In many cases, these different distributions can be modeled as different contexts of a single underlying system, in which each distribution corresponds to a different perturbation of the system, or in causal terms, an intervention. We focus on a class of such causal domain adaptation problems, where data for one or more source domains are given, and the task is to predict the distribution of a certain target variable from measurements of other variables in one or more target domains. We propose an approach for solving these problems that exploits causal inference and does not rely on prior knowledge of the causal graph, the type of interventions or the intervention targets. We demonstrate our approach by evaluating a possible implementation on simulated and real world data.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “Several algorithmic approaches have also been proposed to overcome structural issues, including Peters et al. (2016); Heinze-Deml et al. (2018); Arjovsky et al. (2019); Magliacane et al. (2018).”
An Analysis of the Adaptation Speed of Causal Models (2020, background): “A recent line of work describes these distribution shifts as interventions in an underlying causal model (Zhang et al., 2013; Magliacane et al., 2018).”
Gradient-Based Neural DAG Learning (2019, background): “Causal graphical models (CGM) [12] are BNs which support interventional queries like: What will happen if someone external to the system intervene on variable X? Recent work suggests that causality could partially solve challenges faced by current machine learning systems such as robustness to out-o”
Toward Causal Representation Learning (2021, background): “It is nontrivial (and, in some cases, provably hard [21]) to learn statistical models that are stable across training environments and generalize to novel testing environments [2], [5], [167], [187], [204] drawn from the same environment distribution.”
Towards Causal Representation Learning (2021, background): “in some cases provably hard [20]) to learn statistical models that are stable across training environments and generalize to novel testing environments [189, 204, 167, 5, 2] drawn from the same environment distribution.”
Causal Models for Dynamical Systems (2020, background): “In the i.i.d. case, trade-oﬀs in a similar spirit have been suggested by Rojas-Carulla et al. [2018], Magliacane et al. [2018] and Rothenh¨ausler et al. [2018], for example.”
Causality for Machine Learning (2019, positive): “The line of work provided rather fruitful (Zhang et al., 2013; Weichwald et al., 2014; Zhang et al., 2015; Gong et al., 2016; Huang et al., 2017; Blöbaum et al., 2016; Guo et al., 2018; Wang et al., 2019; Subbaswamy et al., 2018; Rabanser et al., 2018; Li et al., 2018a; Lipton et al., 2018; Li et al”
Invariant Risk Minimization (2019, background): “Some works in machine learning [45, 18, 21, 26, 36, 43, 34, 7] pursue similar questions.”
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms (2019, background): “Magliacane et al. (2018) have sought to identify a subset of features that lead to the best predictions for a variable of interest in a source domain such that the conditional distribution of the variable of interest given these features is the same in the target domain.”
Conditional variance penalties and domain shift robustness (2017, background): “Causal models thus have the defining advantage that the predictions will be valid even under arbitrarily large interventions on all predictor variables (Haavelmo, 1944; Aldrich, 1989; Pearl, 2009; Schölkopf et al., 2012; Peters et al., 2016; Zhang et al., 2013, 2015; Yu et al., 2017; Rojas-Carulla e”
Interventional Few-Shot Learning (2020, methodology): “Causal inference was recently introduced to machine learning [38, 7] and has been applied to various fields in computer vision.”
The Gaussian Graphical Model in Cross-Sectional and Time-Series Data (2016, background): “All network structures may inspire experimental design, or to gather a mixture of observational and experimental data (Magliacane et al., 2017).”
Fairness Violations and Mitigation under Covariate Shift (2019, background): “When causal structure of the domains is known, recent work on causal domain adaptation [38, 61, 66, 68] identify predictors with stable accuracy under unseen changes in distribution.”
Invariant Risk Minimization Games (2020, methodology): “If one has information about the CBN structure, one can find invariant predictors that are maximally predictive using conditional independence tests and other graph-theoretic tools (Magliacane et al., 2018; Subbaswamy et al., 2019).”
Preventing Failures Due to Dataset Shift: Learning Predictive Models That Transport (2018, background): “One solution (see e.g., Subbaswamy and Saria (2018); Magliacane et al. (2018)), which we term graph pruning, is to perform feature selection to find a stable subset of features Z such that the conditional distribution transfers: P (T |Z) = P (T |Z, S).”
Domain Generalization using Causal Matching (2020, methodology): “(Gong et al., 2016; Magliacane et al., 2018; Rojas-Carulla et al., 2018) assume a generative process where the true class label determines the features in the observed data.”
Joint Causal Inference from Multiple Contexts (2016, background): “Because often the true causal structure is not known, while interventional data is available, this requires to extend JCI-based causal discovery with causal prediction techniques, enabling one to predict the results of a particular intervention (Magliacane et al., 2018).”