Causal Transfer in Machine Learning
By Mateo Rojas-Carulla, Bernhard Schölkopf, Richard E. Turner, Jan Peters. 2015
23 citations recorded by the source corpus.
Methods of domain adaptation try to combine knowledge from several related domains (or tasks) to improve performance on a test domain. Inspired by causal methodology, we assume that the covariate shift assumption holds true for a subset of predictor variables: the conditional of the target variable given this subset of predictors is invariant over all tasks. We prove that in an adversarial setting using this subset for prediction is optimal if no examples from the test task are observed. For a specific scenario, in which tasks are drawn from a meta distribution, further optimality results are available. We introduce a practical method which allows for automatic inference of the above subset and provide corresponding code. We present results on synthetic data sets and a gene deletion data set.
Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Causal Inference & Discovery
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Bernhard Schölkopf
- Dominik Janzing
- Kun Zhang
- Dacheng Tao
- Jimeng Sun
- Michel Besserve
- Clark Glymour
- Jan Peters
In Representation & Self-Supervised Learning
- Bernhard Schölkopf
- Dacheng Tao
- Tongliang Liu
- Mingming Gong
- Kun Zhang
- Peng Cui
- Mateo Rojas-Carulla
- Michel Besserve
In Causal Inference & Discovery
- Bernhard Schölkopf
- Kun Zhang
- Dominik Janzing
- Michel Besserve
- Clark Glymour
- Jan Peters
- Krzysztof Chalupka
- Joris M. Mooij
Citations in the corpus
- Learning Independent Causal Mechanisms (2017, background): “Recent applications include semi-supervised learning (Sch¨olkopf et al., 2012) and transfer learning (Rojas-Carulla et al., 2015), in which the authors focus only on linear regression models.”
- Group invariance principles for causal generative models (2017, background): “This includes important cases related to transfer leaning such as covariate-shift or changes in the input distribution [Zhang et al., 2013, 2015; Peters et al., 2016; Bareinboim and Pearl, 2016; Rojas-Carulla et al., 2015], making causal generative models highly relevant in machine learning.”
- Domain Adaptation with Conditional Transferable Components (2016, background): “To this end, some recent work address the domain adaptation problem using causal models to characterize how the distribution changes between domains (Schölkopf et al., 2012; Kun Zhang et al., 2013; 2015; Mateo et al., 2016).”
- Causal Regularization (2017, background): “The interplay between causation and prediction has been studied recently, see (Peters et al., 2015; Rojas-Carulla et al., 2015) and the references therein.”
- Stable Prediction across Unknown Environments (2018, background): “[8] proposed a causal transform framework to learn invariant structure.”