Learning Neural Causal Models from Unknown Interventions

By Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, C. Pal, Yoshua Bengio. 2019

192 citations recorded by the source corpus.

Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data. However, there are theoretical limitations on the identifiability of underlying structures obtained from observational data alone. Interventional data provides much richer information about the underlying data-generating process. However, the extension and application of methods designed for observational data to include interventions is not straightforward and remains an open problem. In this paper we provide a general framework based on continuous optimization and neural networks to create models for the combination of observational and interventional data. The proposed method is even applicable in the challenging and realistic case that the identity of the intervened upon variable is unknown. We examine the proposed method in the setting of graph recovery both de novo and from a partially-known edge set. We establish strong benchmark results on several structure learning tasks, including structure recovery of both synthetic graphs as well as standard graphs from the Bayesian Network Repository.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

In Causal Inference & Discovery

Citations in the corpus

Differentiable Causal Discovery from Interventional Data (2020, methodology): “In some methods, the parameter θ can be partitioned into θ1 and θ2 such that fθ = fθ1 and Aθ = Aθ2 [21, 32, 23] while in others, such a decoupling is not possible, i.”
An Analysis of the Adaptation Speed of Causal Models (2020, background): “This practical challenge opens the door to other suggestions such as inferring the intervention, as explored by Ke et al. (2019).”
Toward Causal Representation Learning (2021, methodology): “It has recently been used for learning causal models [131], modular architectures [29], [85], and disentangled representations [159].”
Towards Causal Representation Learning (2021, methodology): “While [24, 131] focus on learning a causal model using neural networks with an unsupervised loss, the work of Dasgupta et al.”
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms (2019, background): “Experimentally, this generalization of the meta-transfer objective proved to be effective on larger graphs (Ke et al., 2019), in work following the initial release of this paper.”
Inductive biases for deep learning of higher-level cognition (2020, background): “Preliminary work based on meta-learning (Ke et al., 2019; Dasgupta et al., 2019; Bengio et al., 2019) suggests that, parameterizing the correct variables and causal structures, the parameters of the graphical model capturing the (joint) observational distribution can be adapted faster to changes in…”
Fast and Slow Learning of Recurrent Independent Mechanisms (2021, background): “Meta-Learning to Disentangle Causal Mechanisms: Recently (Bengio et al., 2019; Ke et al., 2019) used meta-learning to learn causal mechanisms or causal dependencies between a set of high-level variables, that inspired the approach presented here.”
Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning (2021, methodology): “The sparsity of graphs also impacts the difﬁculty of learning, as observed in [Ke et al., 2019].”
Differentiable Causal Discovery Under Latent Interventions (2022, background): “Recent work by Brouillard et al. (2020); Ke et al. (2020) has shown very promising results in causal discovery by leveraging interventional data with gradient-based learning, even if which variables are intervened (targeted) in each intervention is unknown.”
CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models (2020, background): “[10, 25, 26] explores the structure invariance across multiple domains under interventions to identify causal edges.”
Variational Causal Networks: Approximate Bayesian Inference over Causal Structures (2021, methodology): “While Ke et al. [19] represent each entry of the adjacency matrix as independent Bernoulli variable, such approach is insufﬁcient to capture any dependencies between the entries of the adjacency matrix which are necessary to ensure that q ( A ) only assigns positive probabilities to adjacency…”
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery (2021, background): “Output CMS [152] 2014 low, dynamic/time series NN – no direction NO TEARS [279] 2018 low linear yes no DAG CGNN [74] 2018 low NN yes no DAG SAM [121] 2019 low/medium NN yes no DAG DAG-GNN [272] 2019 low NN yes no DAG GAE [178] 2019 low NN yes no DAG NO BEARS [142] 2019 low/medium/high 3-poly yes no ”
Masked Gradient-Based Causal Structure Learning (2019, methodology): “…meth-ods include Goudet et al. [9] that uses generative neural networks for functional causal modeling with an initial graph skeleton, Ke et al. [13] that uses logistic sigmoid functions to sample binary adjacency matrices in the presence of unknown interventions, Ng et al. [21] that adopts an…”
Causal Discovery from Soft Interventions with Unknown Targets: Characterization and Learning (2020, background): “The works in [4, 23, 38, 30, 15] assume Markovianity (the absence of latent confounders).”