An Analysis of the Adaptation Speed of Causal Models
By Rémi Le Priol, Reza Babanezhad Harikandeh, Yoshua Bengio, Simon Lacoste-Julien. 2020
15 citations recorded by the source corpus.
We consider the problem of discovering the causal process that generated a collection of datasets. We assume that all these datasets were generated by unknown sparse interventions on a structural causal model (SCM) $G$, that we want to identify. Recently, Bengio et al. (2020) argued that among all SCMs, $G$ is the fastest to adapt from one dataset to another, and proposed a meta-learning criterion to identify the causal direction in a two-variable SCM. While the experiments were promising, the theoretical justification was incomplete. Our contribution is a theoretical investigation of the adaptation speed of simple two-variable SCMs. We use convergence rates from stochastic optimization to justify that a relevant proxy for adaptation speed is distance in parameter space after intervention. Using this proxy, we show that the SCM with the correct causal direction is advantaged for categorical and normal cause-effect datasets when the intervention is on the cause variable. When the intervention is on the effect variable, we provide a more nuanced picture which highlights that the fastest-to-adapt heuristic is not always valid. Code to reproduce experiments is available at this https URL
Subfields: Artificial Intelligence, Statistics and Probability, Causal Inference & Discovery
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Bernhard Schölkopf
- Anirudh Goyal
- Stefan Bauer
- Nan Rosemary Ke
- Francesco Locatello
- Kyunghyun Cho
- Quoc V. Le
In Statistics and Probability
- Anirudh Goyal
- Stefan Bauer
- Yoshua Bengio
- Francesco Locatello
- Nan Rosemary Ke
- Nal Kalchbrenner
- Bernhard Schölkopf
- Arthur Gretton
In Causal Inference & Discovery
- Yoshua Bengio
- Anirudh Goyal
- Stefan Bauer
- Francesco Locatello
- Nan Rosemary Ke
- Nal Kalchbrenner
- Bernhard Schölkopf
- Dominik Janzing
Citations in the corpus
- Towards Causal Representation Learning (2021, background): “In certain cases, causal models adapt faster to sparse interventions in distribution [131, 194].”
- Inductive biases for deep learning of higher-level cognition (2020, background): “This comes as a consequence of the fact that fewer parameters need to be adapted as only few variables were affected by the intervention (Priol et al., 2020).”