Causality for Machine Learning

By Bernhard Schölkopf. 2019

567 citations recorded by the source corpus.

Graphical causal inference as pioneered by Judea Pearl arose from research on artificial intelligence (AI), and for a long time had little connection to the field of machine learning. This article discusses where links have been and should be established, introducing key concepts along the way. It argues that the hard open problems of machine learning and AI are intrinsically related to causality, and explains how the field is beginning to understand them.

Subfields: Artificial Intelligence, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Causal Inference & Discovery

Citations in the corpus

Shortcut learning in deep neural networks (2020, background): “Achieving counterfactual explanations of predictions aligned with human intention makes the ultimate goals of adversarial robustness tightly coupled to causality research in machine learning [127].”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “Causal DAGs (Schölkopf, 2019) and explanations (Ross et al.”
An Analysis of the Adaptation Speed of Causal Models (2020, background): “Schölkopf (2019) shed light on these approaches and the broader scope of causality research for machine learning.”
Toward Causal Representation Learning (2021, background): “This could be done by ensuring that they are invariant across problems, exhibit sparse changes to actions or that they can be independently intervened upon [22], [30], [217].”
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “framework, which assumes that test-time queries involve the same distribution as the one that produced the training data (Schölkopf, 2019).”
Towards Causal Representation Learning (2021, background): “The SMS hypothesis was stated in [181, 24, 221, 115], and in earlier form in [218, 279, 220].”
On the Binding Problem in Artificial Neural Networks (2020, background): “As was recently noted by Schölkopf (2019), traditional work in causality assumes given knowledge about the associated causal variables (e.g. objects), and the problem of discovering them (i.e. segregation) has mostly been neglected.”
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning (2020, background): “Moreover, causality has been historically studied from the perspective of probabilistic and causal reasoning (Pearl, 2009), cognitive psychology (Griffiths & Tenenbaum, 2005), and more recently in the context of machine learning (Goyal et al., 2019b; Schölkopf, 2019; Baradel et al., 2019; Bakhtin et”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “A parallel thread of work has studied distribution shifts by identifying changes in causal generative factors (Zhang et al., 2015; 2017; Huang et al., 2017), which is linked to a causal view of disentanglement (Suter et al., 2019; Schölkopf, 2019).”
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects (2020, background): “For XAI, see [1, 2, 31, 55, 61, 67, 75, 111, 120, 128], for causal and counterfactual inference, see [63, 76, 131, 145, 146, 159, 185], and for optimization, see [32, 137, 166].”
ROOTS: Object-Centric Representation and Rendering of 3D Scenes (2020, background): “…is believed to be crucial in enabling various advanced cognitive functions in human-like AI systems (Lake et al., 2017) such as systematic generalization (Bahdanau et al., 2019; van Steenkiste et al., 2019), reasoning (Bottou, 2014), and causal learning (Sch¨olkopf, 2019; Peters et al., 2017).”
Deep Structural Causal Models for Tractable Counterfactual Inference (2020, background): “However, causal approaches have been introduced into deep learning (DL) only very recently [5].”
Deep Learning for Time Series Forecasting: Tutorial and Literature Survey (2020, background): “, [19, 122, 158]) are of particular practical importance in the forecasting setting.”
Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains (2021, background): “Accordingly, causal models aim at creating dependencies between variables only when there exists a causal relationship between them, in order to improve generalization [8].”
CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models (2020, background): “[27] claims the importance and necessity of causal disentangled representation learning but it still remains conceptual.”
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style (2021, unknown): “This is in line with a recently proposed [105] shift towards causal representation learning [40, 76, 85, 87, 106, 107, 112, 123, 127], motivated by the fact that many underlying variables of interest may not be independent but causally related to each other.”
Generative Neurosymbolic Machines (2020, background): “These properties, however, are believed to be crucial in resolving limitations of current deep learning in various System 2 [29] related abilities such as reasoning [6], causal learning [40, 37], accountability [13], and systematic out-of-distribution generalization [3, 46].”