BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery

By Chris Cundy, Aditya Grover, Stefano Ermon. 2021

99 citations recorded by the source corpus.

A structural equation model (SEM) is an effective framework to reason over causal relationships represented via a directed acyclic graph (DAG). Recent advances have enabled effective maximum-likelihood point estimation of DAGs from observational data. However, a point estimate may not accurately capture the uncertainty in inferring the underlying graph in practical scenarios, wherein the true DAG is non-identifiable and/or the observed dataset is limited. We propose Bayesian Causal Discovery Nets (BCD Nets), a variational inference framework for estimating a distribution over DAGs characterizing a linear-Gaussian SEM. Developing a full Bayesian posterior over DAGs is challenging due to the the discrete and combinatorial nature of graphs. We analyse key design choices for scalable VI over DAGs, such as 1) the parametrization of DAGs via an expressive variational family, 2) a continuous relaxation that enables low-variance stochastic optimization, and 3) suitable priors over the latent variables. We provide a series of experiments on real and synthetic data showing that BCD Nets outperform maximum-likelihood methods on standard causal discovery metrics such as structural Hamming distance in low data regimes.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability, Probabilistic & Bayesian Methods, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Simon Lacoste-Julien
2. Yoshua Bengio
3. Stefan Bauer
4. Tristan Deleu
5. Bernhard Schölkopf
6. Chris C. Emezue
7. Peter Richtárik
8. Martin Jaggi

In Control and Systems Engineering

1. Simon Lacoste-Julien
2. Stefan Bauer
3. Yoshua Bengio
4. Tristan Deleu
5. Dougal Maclaurin
6. Fabian Pedregosa
7. Jitendra Malik
8. Ke Li

In Statistics and Probability

1. Simon Lacoste-Julien
2. Stefan Bauer
3. Yoshua Bengio
4. Tristan Deleu
5. Alex Graves
6. Kyunghyun Cho
7. Tijmen Blankevoort
8. Rudolph van der Merwe

In Probabilistic & Bayesian Methods

1. Yoshua Bengio
2. Tristan Deleu
3. Simon Lacoste-Julien
4. Stefan Bauer
5. Chris C. Emezue
6. Ant'onio G'ois
7. Mansi Rankawat
8. Pascal Vincent

In Causal Inference & Discovery

1. Yoshua Bengio
2. Simon Lacoste-Julien
3. Stefan Bauer
4. Tristan Deleu

Citations in the corpus

• Bayesian Structure Learning with Generative Flow Networks (2022, methodology): “Cundy et al. (2021) decomposed the adjacency matrix of a DAG into a triangular matrix and a permutation, and used a continuous relaxation to parametrize a distribution over permutations.”