Towards Causal Representation Learning

By Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, Yoshua Bengio. 2021

361 citations recorded by the source corpus.

The two fields of machine learning and graphical causality arose and developed separately. However, there is now cross-pollination and increasing interest in both fields to benefit from the advances of the other. In the present paper, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.

Subfields: Artificial Intelligence, Management Science and Operations Research, Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Representation & Self-Supervised Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, background): “While there is a significant amount of interest in learning probabilistic or causal graphs between high-level latent variables extracted from low-level observations (Bengio, 2019; Schölkopf, 2019; Schölkopf et al., 2021; Goyal and Bengio, 2021; Ke et al., 2021), there have been comparatively few pra”
Continual Learning by Modeling Intra-Class Variation (2022, background): “, 2017), causality (Schölkopf et al., 2021)) can improve MOCA’s efficacy.”
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style (2021, background): “This is in line with a recently proposed [105] shift towards causal representation learning [40, 76, 85, 87, 106, 107, 112, 123, 127], motivated by the fact that many underlying variables of interest may not be independent but causally related to each other.”
Domain Generalization: A Survey (2021, background): “This is closely related to the topic of causal representation learning, a recent trend in the machine learning community [244].”
AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning (2021, background): “…of only a few variables in the generative processes, so we can just adapt the distribution of a small portion of variables (Huang et al., 2020; Schölkopf et al., 2021) and, furthermore, factorized according to the graph structure, each distribution module can be adapted separately…”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, background): “Causal Models are intermediate descriptions that abstract physical models while answering statistical model questions (Schölkopf et al, 2021).”