Justifying Additive Noise Model-Based Causal Discovery via Algorithmic Information Theory

By Dominik Janzing, Bastian Steudel. 2009

35 citations recorded by the source corpus.

A recent method for causal discovery is in many cases able to infer whether X causes Y or Y causes X for just two observed variables X and Y. It is based on the observation that there exist (non-Gaussian) joint distributions P(X,Y) for which Y may be written as a function of X up to an additive noise term that is independent of X and no such model exists from Y to X. Whenever this is the case, one prefers the causal model X → Y. Here we justify this method by showing that the causal hypothesis Y → X is unlikely because it requires a specific tuning between P(Y) and P(X|Y) to generate a distribution that admits an additive noise model from X to Y. To quantify the amount of tuning, needed we derive lower bounds on the algorithmic information shared by P(Y) and P(X|Y). This way, our justification is consistent with recent approaches for using algorithmic information theory for causal reasoning. We extend this principle to the case where P(X,Y)almost admits an additive noise model. Our results suggest that the above conclusion is more reliable if the complexity of P(Y) is high.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Identifying Cause and Effect on Discrete Data using Additive Noise Models (2010, background): “Janzing & Steudel (2009) give further theoretical support for this principle of causal inference using the concept of Kolmogorov complexity and Peters et al. (2009a) use this principle to detect whether a sample of a time series has been reversed.”
Origo: causal inference by compression (2016, methodology): “It has also been used in justifying the additive noise model based causal discovery [24].”
UvA-DARE ( Digital Academic Repository ) Causal Discovery with Continuous Additive Noise Models (2014, unknown): “ and X 2 and forward and backward functions. Proof See Appendix A.9. Example 24 shows how parameters of function, input and noise distribution have to be \ne-tuned" to yield non-identiability (Janzing and Steudel, 2010). It can be shown that bivariate identiability even holds generically when c”
Information-geometric approach to inferring causal directions (2012, background): “The reason is that this would require rather contrived adjustments between P X (the marginal distribution of the hypothetical cause) and P Y |X (the conditional distribution of the effect, given the cause) [12].”
Causal Inference on Time Series using Restricted Structural Equation Models (2013, background): “This, however, requires an " unnatural " fine tuning of the functions [Janzing and Steudel, 2010] and is relevant only when there are time series without time structure or the data are non-faithful (see Theorem 1).”
Causal Inference on Time Series using Structural Equation Models (2012, background): “This requires an “unnatural” ﬁne tuning of the functions and Janzing and Steudel [2010] argue that in the case of causal suﬃciency it cannot occur if one believes in the “independence” of cause and causal mechanism.”
Causal Inference by Stochastic Complexity (2017, methodology): “Altogether we can compute the multinomial stochastic complexity in O(n).”
Identiﬁability of Causal Graphs using Functional Models (background): “We would like to argue, however, that this case is unlikely in the following sense: Janzing and Steudel [2010] use the concept of Kolmogorov complexity to show that it can only happen if the cause distribution p (cause) and the conditional distribution of the eﬀect given the cause p (eﬀect | cause)…”