Invariant Models for Causal Transfer Learning

By Mateo Rojas-Carulla, Bernhard Schölkopf, Richard E. Turner, Jan Peters. 2015

443 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Causal Inference & Discovery

Citations in the corpus

Toward Causal Representation Learning (2021, methodology): “To formalize this general intuition, one can consider a form of out-of-distribution generalization, which can be optimized by minimizing the empirical risk over a class of distributions induced by a causal model of the data [5], [169], [187], [204], [220].”
Towards Causal Representation Learning (2021, background): “in some cases provably hard [20]) to learn statistical models that are stable across training environments and generalize to novel testing environments [189, 204, 167, 5, 2] drawn from the same environment distribution.”
Learning explanations that are hard to vary (2020, background): “…2012].14 Based on this insight, it was proposed that regression based on causal features should presents desirable invariance and robustness properties [Mooij et al., 2009, Schölkopf et al., 2012, Peters et al., 2016, Rojas-Carulla et al., 2018, Heinze-Deml et al., 2018, von Kügelgen et al., 2018].”
Causality for Machine Learning (2019, background): “The line of work provided rather fruitful (Zhang et al., 2013; Weichwald et al., 2014; Zhang et al., 2015; Gong et al., 2016; Huang et al., 2017; Blöbaum et al., 2016; Guo et al., 2018; Wang et al., 2019; Subbaswamy et al., 2018; Rabanser et al., 2018; Li et al., 2018a; Lipton et al., 2018; Li et al”
Learning Neural Causal Models from Unknown Interventions (2019, background): “[39] exploit invariance across environments to infer causal structure, which faces difficulty scaling due to the iteration over the super-exponential set of possible graphs.”
Invariant Risk Minimization (2019, background): “Some works in machine learning [45, 18, 21, 26, 36, 43, 34, 7] pursue similar questions.”
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms (2019, methodology): “Rojas-Carulla et al. (2018) propose a method to ﬁnd an optimal subset that makes the target independent from the selection variables.”
Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness (2018, background): “It has been proposed that features should be selected based on their robustness or invariance across tasks (e.g., Rojas-Carulla et al., 2018), we hence do not want them to be affected by changes in any other factor.”
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision (2020, background): “on-and-language [24] and NLP [78] have been shown to exhibit strong biases, leading to an in ated sense of progress on these tasks. Recent works have discussed generalization from a causal perspective[6,29,50,54]. This sheds light on the possible avenues for addressing the issue. In order to general”
Fairness Violations and Mitigation under Covariate Shift (2019, methodology): “Manymethods exist to learn under distribution shifts [58], including recent work from a causal inference perspective [2, 51, 61, 68].”
The Risks of Invariant Risk Minimization (2020, background): “Various works on invariant prediction (Muandet et al., 2013; Ghassami et al., 2017; Heinze-Deml et al., 2018; Rojas-Carulla et al., 2018; Subbaswamy et al., 2019; Christiansen et al., 2020) consider regression in both the linear and non-linear setting, but they exclusively focus on learning with…”
Domain Generalization using Causal Matching (2020, methodology): “There is work on use of causal reasoning for domain adaptation (Gong et al., 2016; Heinze-Deml & Meinshausen, 2019; Magliacane et al., 2018; Rojas-Carulla et al., 2018) that assumes Y → X direction and other work (Arjovsky et al.”
A Review of Domain Adaptation without Target Labels (2019, background): “assumptions, such as covariate shift or prior shift, are reasonable [82], [210].”
Towards Out-Of-Distribution Generalization: A Survey (2021, methodology): “Under the controllability condition proposed by Koyama et al. [140] which assumes that there exists an environment e such that X ⊥ Y | Φ( X ) , e , the optimality of E [ Y | Φ( X )] can be veriﬁed, as shown in Proposition 3.1 in [140] and Theorem 1 in [158].”
Explaining The Efficacy of Counterfactually-Augmented Data (2020, background): “where the relationship between a target variable and its direct causal ancestors remains invariant across foreseeable shifts (Peters et al., 2016; Ghassami et al., 2017; Rojas-Carulla et al., 2018; Kuang et al., 2018; Magliacane et al., 2018; Christiansen & Peters, 2020; Weichwald & Peters, 2020).”
Interventional Robustness of Deep Latent Variable Models (2018, background): “Since features are selected based on their expressiveness for causal factors that are relevant for a specific task (e.g. Rojas-Carulla et al., 2018), we specifically want them not to be affected by changes in any other factor.”