Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks

By Joris M. Mooij, Jan Peters, Dominik Janzing, Jakob Zscheischler, Bernhard Schölkopf. 2014

150 citations recorded by the source corpus.

The discovery of causal relationships from purely observational data is a fundamental problem in science. The most elementary form of such a causal discovery problem is to decide whether X causes Y or, alternatively, Y causes X, given joint observations of two variables X,Y. An example is to decide whether altitude causes temperature, or vice versa, given only joint measurements of both variables. Even under the simplifying assumptions of no confounding, no feedback loops, and no selection bias, such bivariate causal discovery problems are challenging. Nevertheless, several approaches for addressing those problems have been proposed in recent years. We review two families of such methods: methods based on Additive Noise Models (ANMs) and Information Geometric Causal Inference (IGCI). We present the benchmark CAUSEEFFECTPAIRS that consists of data for 100 different causee ffect pairs selected from 37 data sets from various domains (e.g., meteorology, biology, medicine, engineering, economy, etc.) and motivate our decisions regarding the "ground truth" causal directions of all pairs. We evaluate the performance of several bivariate causal discovery methods on these real-world benchmark data and in addition on artificially simulated data. Our empirical results on real-world data indicate that certain methods are indeed able to distinguish cause from effect using only purely observational data, although more benchmark data would be needed to obtain statistically significant conclusions. One of the best performing methods overall is the method based on Additive Noise Models that has originally been proposed by Hoyer et al. (2009), which obtains an accuracy of 63 ± 10 % and an AUC of 0.74 ± 0.05 on the real-world benchmark. As the main theoretical contribution of this work we prove the consistency of that method.

Subfields: Artificial Intelligence, Statistics and Probability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Causal Inference & Discovery

Citations in the corpus

Causality for Machine Learning (2019, positive): “We have collected a set of benchmark problems for cause-effect inference, and by now there is a number of methods that can detect causal direction better than chance (Mooij et al., 2016), some of them building on the above Kolmogorov complexity model (Budhathoki and Vreeken, 2016), and some directly”
The Arrow of Time in Multivariate Time Series (2016, background): “This result is not immediately straightforward since the independence measure is based on estimated residuals rather than “true” innovations (see also Mooij et al., 2014).”
Revisiting Classifier Two-Sample Tests (2016, methodology): “andom variables, denoted by Xand Y. That is, given the sample D= f(x i;y i)gn i=1 ˘P n(X;Y), our goal is to conclude whether “Xcauses Y”, or “Y causes X”. We call this problem cause-effect discovery (Mooij et al., 2016). In the case where X!Y, we can write the cause-effect relationship as: x˘P(X); n”
COPHY: Counterfactual Learning of Physical Dynamics (2019, background): “…gained mainstream attention relatively recently (Lopez-Paz et al., 2017; Lopez-Paz & Oquab, 2017; Kocaoglu et al., 2018; Rojas-Carulla et al., 2018; Mooij et al., 2016; Sch¨olkopf et al., 2012), due to limitations of statistical learning becoming increasingly apparent (Pearl, 2018; Lake et al.,…”
The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data (2019, background): “The cause-effect pairs challenge [Mooij et al., 2016] provides data that is empirical and, while interventional effects are not available, the direction of causality is known.”
Towards a Learning Theory of Cause-Effect Inference (2015, unknown): “0 20 40 60 80 100 0 20 40 60 80 100 decission rate classification accuracy RCC ANM IGCI Figure 1 .”
Identifying Best Interventions through Online Importance Sampling (2017, background): “There has been a lot of work [27, 16, 12, 15, 35, 29, 24, 33] on learning casual models from data and/or experiments and using it to estimate causal strength questions of the counterfactual nature.”
Causal Inference via Kernel Deviance Measures (2018, unknown): “Given the growing interest in using data to guide decisions in areas where interventional and counterfactual questions abound, causal discovery methods have attracted considerable research interest [Hoyer et al., 2009, Zhang and Hyv¨arinen, 2009, Lopez-Paz et al., 2015, Mooij et al., 2016].”
Explainable and Interpretable Models in Computer Vision and Machine Learning (2017, unknown): “ANM-HSIC yields good results when the additive assumption holds (e.g. on CE-Gauss ), but fails otherwise.”
Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models (2019, background): “Assumptions on structural mechanisms are also used in structure learning to identify the direction of causation X → Y or X ← Y from the observational distribution alone, with both continuous (Peters & Schölkopf, 2014; Mooij et al., 2016) and discrete (Kocaoglu et al., 2017) variables.”
A Survey of Learning Causality with Data (2018, background): “A special case is the bivariate data and the task reduces to distinguishing the cause from the effect [104].”
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery (2021, methodology): “We now briefly provide some examples of structural asymmetries, and direct interested readers to Mooij et al. (2016) [166] for a detailed review.”