Telling Cause from Effect Using MDL-Based Local and Global Regression

By Alexander Marx, Jilles Vreeken. 2017

71 citations recorded by the source corpus.

We consider the fundamental problem of inferring the causal direction between two univariate numeric random variables X and Y from observational data. The two-variable case is especially difficult to solve since it is not possible to use standard conditional independence tests between the variables. To tackle this problem, we follow an information theoretic approach based on Kolmogorov complexity and use the Minimum Description Length (MDL) principle to provide a practical solution. In particular, we propose a compression scheme to encode local and global functional relations using MDL-based regression. We infer X causes Y in case it is shorter to describe Y as a function of X than the inverse direction. In addition, we introduce Slope, an efficient linear-time algorithm that through thorough empirical evaluation on both synthetic and real world data we show outperforms the state of the art by a wide margin.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Molecular Biology

Citations in the corpus

Generalization in anti-causal learning (2018, methodology): “Practical approaches based on MDL as a proxy for Kolmogorov complexity can successfully identify causal directions from data (Budhathoki and Vreeken, 2016; Marx and Vreeken, 2017), providing evidence that the anti-causal direction has higher expression complexity.”
Causal Reasoning and Large Language Models: Opening a New Frontier for Causality (2023, background): “Slope (Marx & Vreeken, 2017) 0.75 0.83 bQCD (Tagasovska et al., 2020) 0.68 0.75 PNL-MLP (Zhang & Hyvarinen, 2012) From these facts, we can determine that a decrease in ozone concentration would lead to an increase in radiation.”
Structural Agnostic Modeling: Adversarial Learning of Causal Graphs (2018, methodology): “e Working Hypothesis 1 and the causal Markov and faithfulness assumptions: Xd j=1 K(p(x jjx Pa( ;G))) + Xd j=1 K(p(x jjx Pa(j;Gb))); (23) Proof in Appendix C Following (Janzing and Scholkopf, 2010; Marx and Vreeken, 2017) and approximating the Kolmogorov complexity with the Minimum Description Leng”
Cause-Effect Inference by Comparing Regression Errors (2018, background): “[24], which appeared after submission, also infers the causal direction via regression, but from a minimum description length perspective.”