Invariant Causal Prediction for Nonlinear Models

By Christina Heinze-Deml, Jan Peters, Nicolai Meinshausen. 2017

315 citations recorded by the source corpus.

Abstract An important problem in many domains is to predict how a system will respond to interventions. This task is inherently linked to estimating the system’s underlying causal structure. To this end, Invariant Causal Prediction (ICP) [1] has been proposed which learns a causal model exploiting the invariance of causal relations using data from different environments. When considering linear models, the implementation of ICP is relatively straightforward. However, the nonlinear case is more challenging due to the difficulty of performing nonparametric tests for conditional independence. In this work, we present and evaluate an array of methods for nonlinear and nonparametric versions of ICP for learning the causal parents of given target variables. We find that an approach which first fits a nonlinear model with data pooled over all environments and then tests for differences between the residual distributions across environments is quite robust across a large variety of simulation settings. We call this procedure “invariant residual distribution test”. In general, we observe that the performance of all approaches is critically dependent on the true (unknown) causal structure and it becomes challenging to achieve high power if the parental set includes more than two variables. As a real-world example, we consider fertility rate modeling which is central to world population projections. We explore predicting the effect of hypothetical interventions using the accepted models from nonlinear ICP. The results reaffirm the previously observed central causal role of child mortality rates.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “Several algorithmic approaches have also been proposed to overcome structural issues, including Peters et al. (2016); Heinze-Deml et al. (2018); Arjovsky et al. (2019); Magliacane et al. (2018).”
Differentiable Causal Discovery from Interventional Data (2020, methodology): “Another type of constraint-based method exploits the invariance of causal mechanisms across interventional distributions, e.g., ICP [27, 14].”
An Analysis of the Adaptation Speed of Causal Models (2020, background): “This work has set a fruitful line of research for robust machine learning (Heinze-Deml et al., 2018b,a; Rothenhäusler et al., 2019; Arjovsky et al., 2019).”
Toward Causal Representation Learning (2021, background): “…[153], [156], [167], [195], [204], [243], [263], [267], [277], [278], [281], complementing the studies of Bareinboim and Pearl [14], [185], and it inspired a thread of work in the statistics community exploiting invariance for causal discovery and other tasks [105], [106], [114], [187], [191].”
Towards Causal Representation Learning (2021, background): “263, 243, 195, 152, 156, 153, 167, 204, 115], complementing the studies of Bareinboim and Pearl [12, 185], and it inspired a thread of work in the statistics community exploiting invariance for causal discovery and other tasks [189, 192, 105, 104, 115].”
Learning explanations that are hard to vary (2020, background): “…2012].14 Based on this insight, it was proposed that regression based on causal features should presents desirable invariance and robustness properties [Mooij et al., 2009, Schölkopf et al., 2012, Peters et al., 2016, Rojas-Carulla et al., 2018, Heinze-Deml et al., 2018, von Kügelgen et al., 2018].”
Causality for Machine Learning (2019, background): “in Statistics at ETH, he carried on with the invariance idea, leading to a thread of work in the statistics community exploiting invariance for causal discovery and other tasks (Peters et al., 2016; Pfister et al., 2018a; Heinze-Deml et al., 2017; Heinze-Deml and Meinshausen, 2017).”
Learning Neural Causal Models from Unknown Interventions (2019, methodology): “Table 1: Baseline comparisons: Structural Hamming Distance (SHD) (lower is better) for learned and ground-truth edges on various graphs from both synthetic and real datasets, compared to [37], [51], [18], [15] and [14].”
Invariant Risk Minimization (2019, background): “As shown by [40, 22], there exists an intimate link between invariance and causation useful for generalization.”
Causal Regularization (2019, background): “It is believed (and can be proven subject to appropriate model assumptions) that finding statistical models that generalize well across different background conditions is closely linked to finding causal models [7, 8, 9, 10].”
The hardness of conditional independence testing and the generalised covariance measure (2018, methodology): “In the following, we compare the GCM with three other conditional independence tests: KCI [51] with its implementation from CondIndTests [26], and the residual prediction test [26, 46].”
Characterizing and Learning Equivalence Classes of Causal DAGs under Interventions (2018, methodology): “Testing for invariance: To test whether a (conditional) distribution f ( I ) ( X i | X S ) is invariant, we used a method proposed by Heinze-Deml et al. (2017) that we found to work well in practice.”
Observational Overfitting in Reinforcement Learning (2019, background): “This makes a more explicit connection to causal inference (Arjovsky et al., 2019; Heinze-Deml & Meinshausen, 2019; Heinze-Deml et al., 2019) where spurious correlations between ungeneralizable features and progress may make training easy, but are detrimental to test performance because they induce…”
Out-of-Distribution Generalization via Risk Extrapolation (REx) (2020, background): “ICP and its nonlinear extension (Heinze-Deml et al., 2018) use statistical tests to detect whether the residuals of a linear model are equal across domains.”
Variational Causal Networks: Approximate Bayesian Inference over Causal Structures (2021, unknown): “[16] extend ICP to non-linear SCMs.”
PHYRE: A New Benchmark for Physical Reasoning (2019, background): “Methods aimed at invariant causal prediction (ICP) [14, 38] may be well-suited for PHYRE.”