Causal Inference on Discrete Data Using Additive Noise Models

By Jan Peters, Dominik Janzing, Bernhard Schölkopf. 2009

166 citations recorded by the source corpus.

Inferring the causal structure of a set of random variables from a finite sample of the joint distribution is an important problem in science. The case of two random variables is particularly challenging since no (conditional) independences can be exploited. Recent methods that are based on additive noise models suggest the following principle: Whenever the joint distribution P(X,Y) admits such a model in one direction, e.g., Y = f(X)+N, N ⊥X, but does not admit the reversed model X=g(Y)+Ñ, Ñ ⊥ Y, one infers the former direction to be causal (i.e., X → Y). Up to now, these approaches only dealt with continuous variables. In many situations, however, the variables of interest are discrete or even have only finitely many states. In this work, we extend the notion of additive noise models to these cases. We prove that it almost never occurs that additive noise models can be fit in both directions. We further propose an efficient algorithm that is able to perform this way of causal inference on finite samples of discrete variables. We show that the algorithm works on both synthetic and real data sets.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistics and Probability

In Causal Inference & Discovery

Citations in the corpus

Gradient-Based Neural DAG Learning (2019, background): “There are many examples of graph identifiability results for continuous variables [11, 9, 14, 17] as well as for discrete variables [10].”
Kernel‐based tests for joint independence (2016, methodology): “A more efficient method, which is based on a similar idea, is the regression with subsequent independence test algorithm that was described in Peters et al. (2014), section 4.1. Also the computationally efficient causal additive model method (Bühlmann et al., 2014) could be equipped with a joint ind”
Consistency of Causal Inference under the Additive Noise Model (2013, methodology): “Various works (Shimizu et al., 2006; Hoyer et al., 2009; Peters et al., 2011a) have successfully validated procedures based on the CAM (outlined in Section 1.1 below) on a mix of artificial and real-world datasets where the causal structure to be inferred is clear.”
Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, background): “Without having access to the interventional distributions, this symmetry prevents us from drawing any conclusions regarding the direction of the causal relationship between X and Y if we only have access to the observational distribution PX,Y .”
Estimating Causal Direction and Confounding of Two Discrete Variables (2016, methodology): “The additive noise models framework was applied to discrete variables by Peters et al. (2011).”
Causal Inference on Time Series using Structural Equation Models (2012, background): “This idea has led to the extensions of nonlinear additive functions f ( x, n ) = g ( x ) + n [Hoyer et al., 2009], post-nonlinear additive functions f ( x, n ) = h g ( x ) + n [Zhang and Hyvarinen, 2009] and discrete functions [Peters et al., 2011a].”
Robust Learning via Cause-Effect Models (2011, background): “A similar statement has been shown for discrete data [10] and for the post-nonlinear ANM model [11] where ψ is an invertible function.”
Cause-Effect Inference by Comparing Regression Errors (2018, background): “In particular, the identification of the causal relation between only two observed variables is a challenging task [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15].”
Causal Inference via Algebraic Geometry: Feasibility Tests for Functional Causal Structures with Two Binary Observed Variables (2015, background): “[5].”
Information-geometric approach to inferring causal directions (2012, background): “[9,10] described a potential asymmetry between cause and effect where independence is meant in terms of statistical independence between the cause and the noise term that occurs in the causal mechanism: If Y is a function of X up to an additive noise term that is statistically independent of X , i.”
From Dependence to Causation (2016, background): “Additive noise models are consistent (Kpotufe et al., 2014), and there exists extensions to discrete variables (Peters et al., 2011), latent variables (Stegle et al., 2010), cyclic graphs (Mooij et al., 2011; Lacerda et al., 2012), and postnonlinear equations xi = gi(fi(Pa(xi)) + ni), where gi : R→…”
A Sound and Complete Algorithm for Learning Causal Models from Relational Data (2013, methodology): “Finally, orienting bivariate dependencies, the most effective constraint used by RCD, is similar to the eﬀorts of Shimizu et al. (2006), Hoyer et al. (2008), and Peters et al. (2011) in the propositional setting.”
Identifying Cause and Effect on Discrete Data using Additive Noise Models (2010, background): “Janzing & Steudel (2009) give further theoretical support for this principle of causal inference using the concept of Kolmogorov complexity and Peters et al. (2009a) use this principle to detect whether a sample of a time series has been reversed.”
Causal Inference on Time Series using Restricted Structural Equation Models (2013, background): “For large sample sizes, even smallest differences between the true data generating process and the model may lead to rejected independence tests [discussed by Peters et al., 2011a] .”
Quantifying causal influences (2012, background): “BALDUZZI, GROSSE-WENTRUP AND SCHOLKOPF¨ 7.1. DAGs without time structure. We here restrict attention to linear structural equations, but interesting generalizations are given by additive noise models [8, 17, 18] and post-nonlinear models [23]. The ﬁrst step in estimating the causal strength consists”
Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, background): “…y) satisfies an Additive Noise Model X → Y , or an Additive Noise Model Y → X, we simply estimate the regression functions in both directions, calculate the corresponding residuals, estimate the dependence of the residuals with respect to the input by some dependence measure Ĉ, and output the…”
On Azadkia–Chatterjee’s conditional dependence coefficient (2021, background): “…and Cai et al. (2022) (based on Rosenblatt transformation); Bergsma (2004, 2011) and Veraverbeke et al. (2011) (copula-based); Hoyer et al. (2009), Peters et al. (2011), Shah and Peters (2020), and Petersen and Hansen (2021) (regression-based); Canonne et al. (2018) and Neykov et al. (2021)…”
UvA-DARE ( Digital Academic Repository ) Causal Discovery with Continuous Additive Noise Models (2014, unknown): “This paper builds on the conference papers of Hoyer et al. (2009), Peters et al. (2011b) and Mooij et al. (2009) 1 but extends the material in several aspects.”