CAM: Causal Additive Models, high-dimensional order search and penalized regression

By Peter Buhlmann, Jan Peters, Jan Ernest. 2013

405 citations recorded by the source corpus.

We develop estimation for potentially high-dimensional additive structural equation models. A key component of our approach is to decouple order search among the variables from feature or edge selection in a directed acyclic graph encoding the causal structure. We show that the former can be done with nonregularized (restricted) maximum likelihood estimation while the latter can be efficiently addressed using sparse regression techniques. Thus, we substantially simplify the problem of structure search and estimation for an important class of causal models. We establish consistency of the (restricted) maximum likelihood estimator for low- and high-dimensional scenarios, and we also allow for misspecification of the error distribution. Furthermore, we develop an efficient computational algorithm which can deal with many variables, and the new method’s accuracy and performance is illustrated on simulated and real data.

Subfields: Statistics and Probability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Causal Inference & Discovery

Citations in the corpus

Differentiable Causal Discovery from Interventional Data (2020, methodology): “For all data sets, the source nodes are Gaussian with zero mean and variance sampled from U [1 , 2] .”
Gradient-Based Neural DAG Learning (2019, methodology): “Most score-based methods embrace the combinatorial nature of the problem via greedy search procedures [3, 2].”
Invariant Causal Prediction for Sequential Data (2017, methodology): “Many procedures in this field only use observational data (e.g., Spirtes, Glymour, and Scheines 2000; Chickering 2002; Shimizu et al. 2006; Janzing et al. 2012; van de Geer and Bühlmann 2013; Peters et al. 2014; Bühlmann, Peters, and Ernest 2014).”
Causal inference by using invariant prediction: identification and confidence intervals (2015, background): “, 2006), non-linearities (Hoyer et al., 2009; Peters et al., 2014; Bühlmann et al., 2014), equal error variances (Peters and Bühlmann, 2014) or shared information between the regression function and target variable (Janzing et al.”
Consistency of Causal Inference under the Additive Noise Model (2013, background): “In a recent Arxived result appearing after our initial submission, Buhlmann et al. (2013) showed the consistency of a maximum log-likelihood approach to causal inference under the multi-variable network extension of Peters et al. (2011b).”
Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, methodology): “For the Gaussian case, that is equivalent to the score considered in a high-dimensional context that was shown to be consistent by Bühlmann et al. (2014).”
Learning Sparse Nonparametric DAGs (2019, methodology): “We first observe that NOTEARS-MLP outperforms CAM in multiple index models and MLP models, on the other hand, CAM achieves better accuracy on additive GP and the full GP setting.”
Structure Learning in Graphical Modeling (2016, background): “Identiﬁability of the DAG can also be achieved by various other restrictions on SEMs with additive noise, such as non-linear structural equations (Hoyer et al., 2009), additive models (B¨uhlmann et al., 2014), or equal error variances (Peters & B¨uhlmann, 2014).”
Causal inference in partially linear structural equation models (2016, methodology): “Moreover, we provide some robustness results for identifiability in the neighborhood of Gaussian noise and we derive (high-dimensional) consistency based on the consistency proof of the CAM methodology in [3].”
High-dimensional learning of linear causal networks via inverse covariance estimation (2013, methodology): “Bühlmann et al. (2013) isolate sufficient conditions under which Lasso-based linear regression could be used to recover a conditional independence graph for general distributions, and use it as a prescreening step for nonparametric causal inference in additive noise models; however, it is unclear…”
Generalized Score Functions for Causal Discovery (2018, methodology): “More recently, some other score-based methods [2, 4, 18, 19, 26] and hybrid methods [9, 31] have been proposed; they exploit some assumptions of the model class.”
Learning Directed Acyclic Graphs with Penalized Neighbourhood Regression (2015, methodology): “B¨uhlmann et al. (2014) propose a three-step method for learning nonlinear SEM that uses preliminary neighbourhood search, order search, and ﬁnally sparse additive regression.”
Causal Discovery Toolbox: Uncover causal relationships in Python (2019, methodology): “…conditional independence tests named constraint-based methods , such as PC or FCI (Spirtes et al., 2000; Strobl et al., 2017), or on score-based methods , ﬁnding the graph that maximizes a likelihood score through graph search heuristics, like GES or CAM (Chickering, 2002; B¨uhlmann et al., 2014).”
Repair of Partly Misspecified Causal Diagrams. (2017, methodology): “25 The proposed vetting method applies in cases (C1) and (C2).”
Structural Agnostic Modeling: Adversarial Learning of Causal Graphs (2018, methodology): “This is why CAM obtain most of the time the best results in Figure 24 for this type of graph.”
On the uniform convergence of empirical norms and inner products, with application to causal inference (2013, methodology): “For more details we refer to Bühlmann et al. [2013]. The results can also be applied to generalize the results in van de Geer and Bühlmann [2013] for DAG’s to the linear non-Gaussian case, in particular to anisotropic distributions. A generalization to to isotropic distributions (e.g. sub-Gaussian d”