Learning Sparse Nonparametric DAGs

By Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, Eric P. Xing. 2019

339 citations recorded by the source corpus.

We develop a framework for learning sparse nonparametric directed acyclic graphs (DAGs) from data. Our approach is based on a recent algebraic characterization of DAGs that led to a fully continuous program for score-based learning of DAG models parametrized by a linear structural equation model (SEM). We extend this algebraic characterization to nonparametric SEM by leveraging nonparametric sparsity based on partial derivatives, resulting in a continuous optimization problem that can be applied to a variety of nonparametric and semiparametric models including GLMs, additive noise models, and index models as special cases. Unlike existing approaches that require specific modeling choices, loss functions, or algorithms, we present a completely general framework that can be applied to general nonlinear models (e.g. without additive noise), general differentiable loss functions, and generic black-box optimization routines. The code is available at this https URL.

Subfields: Artificial Intelligence, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Causal Inference & Discovery

Citations in the corpus

Differentiable Causal Discovery from Interventional Data (2020, methodology): “Several other methods use neural networks to model nonlinear relations via fθ and have been shown to be competitive with classical methods [27, 53].”
DiBS: Differentiable Bayesian Structure Learning (2021, methodology): “Our setup follows [9, 24, 27, 52], who consider inferring BNs with Erdős-Rényi and scale-free random structures [53, 54].”
Differentiable Causal Discovery Under Latent Interventions (2022, methodology): “Several other methods apply non-linear models such as neural networks (Lachapelle et al., 2019; Zheng et al., 2020) and define W G differently.”
Masked Gradient-Based Causal Structure Learning (2019, methodology): “…PC-KCI (with kernel-based conditional independence tests) [40], GES [5], GES-GS (with generalized score) [10], CAM [3], NOTEARS [43], DAG-GNN [37], SAM [12], NOTEARS-MLP [44], and For synthetic data, experiments are conducted on data models varying along graph size, degree, and causal relationship.”
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery (2021, methodology): “) [138] 2020 low/medium NN yes no DAG GranDAG [139] 2020 low NN yes no DAG MaskedNN [176] 2020 low NN yes no DAG CausalVAE [267] 2020 image NN yes yes DAG CAREFL [126] 2020 low NN yes no DAG / direction Varando [251] 2020 low linear yes no DAG NO TEARS+ [280] 2020 low non-linear yes no DAG ICL [258]”
DYNOTEARS: Structure Learning from Time-Series Data (2020, methodology): “For example, using the general approach outlined in Zheng et al. (2019), it is possible to model complicated nonlinear dependencies via neural networks or orthogonal basis expansions.”
Knowledge Augmented Deep Neural Networks for Joint Facial Expression and Action Unit Recognition (2020, background): “With weights w = {wijk}, the structure of rBN is parameterized as a weighted adjacency matrix 2 A ∈ RN×N with Aij = ∑K k=1 ||wijk||(2)2([58]).”