Toward Causal Representation Learning
By Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, Yoshua Bengio. Proceedings of the IEEE, 2021
1,515 citations recorded by the source corpus.
The two fields of machine learning and graphical causality arose and are developed separately. However, there is, now, cross-pollination and increasing interest in both fields to benefit from the advances of the other. In this article, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, that is, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.
Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Causal Inference & Discovery
View Paper: https://doi.org/10.1109/JPROC.2021.3058954
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Bernhard Schölkopf
- Anirudh Goyal
- Mohit Bansal
- Sergey Levine
- Quoc V. Le
- Kyunghyun Cho
- Nan Rosemary Ke
In Representation & Self-Supervised Learning
- Anirudh Goyal
- Yoshua Bengio
- Bernhard Schölkopf
- Stefan Bauer
- Nan Rosemary Ke
- C. Pal
- Michael C. Mozer
- Danilo Jimenez Rezende
In Causal Inference & Discovery
Citations in the corpus
- Conterfactual Generative Zero-Shot Semantic Segmentation (2021, methodology): “This is called Independent Causal Mechanism (ICM) Principle [29].”
- Towards Out-Of-Distribution Generalization: A Survey (2021, background): “…Learning ( Section 3.1 ) [27], [28], [29], [30], [31], [32], [33](Section 3.1) Causal Representation Learning ( Section 3.2 ) [34], [35], [36](Section 3.2) SupervisedLearningModels ( Section 4 ) ( Section 4.1 ) [7], [8], [9], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46],…”
- AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning (2021, background): “…of only a few variables in the generative processes, so we can just adapt the distribution of a small portion of variables (Huang et al., 2020; Schölkopf et al., 2021) and, furthermore, factorized according to the graph structure, each distribution module can be adapted separately…”
- Artificial intelligence and illusions of understanding in scientific research (2024, neutral): “For instance, researchers are actively working towards developing AI models that have human-like causal learning and reasoning capabilities 183–185 (although many hurdles remain before this is achieved 186 ).”
- Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language (2022, background): “However, these interpretations can be reconciled by the argument that non-causal features are inherently unlikely to be robust, which is sometimes formalized as the principle of sparse mechanism shift (Schölkopf et al., 2021).”
- Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning (2021, background): “As a result, there has been a recent surge of interest in learning causal models for deep reinforcement learning [de Haan et al., 2019, Dasgupta et al., 2019, Nair et al., 2019, Goyal et al., 2019, Goyal and Bengio, 2020, Rezende et al., 2020, Wang et al., 2021, Schölkopf et al., 2021].”
- Eigenfunction Extraction for Ordered Representation Learning (2025, unknown): “An alternative modeling of the representation space is proposed by the causal representation learning literature, which aims to form a causal understanding of the world by learning appropriate representations that support causal interventions, reasoning, and planning (Schölkopf et al., 2021).”
- HyQE: Ranking Contexts with Hypothetical Query Embeddings (2024, background): “It is widely known that generative models that respect the causality relationships are more robust to distribution shifts because they can avoid learning spurious relationships between random variables (Ahuja et al., 2021; Schölkopf et al., 2021; Lu et al., 2022).”