On causal and anticausal learning

By Bernhard Schölkopf, Dominik Janzing, Jan Peters, Eleni Sgouritsa, Kun Zhang, Joris M. Mooij. 2012

682 citations recorded by the source corpus.

We consider the problem of function estimation in the case where an underlying causal model can be inferred. This has implications for popular scenarios such as covariate shift, concept drift, transfer learning and semi-supervised learning. We argue that causal knowledge may facilitate some approaches for a given problem, and rule out others. In particular, we formulate a hypothesis for when semi-supervised learning can help, and corroborate it with empirical results.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

Citations in the corpus

Shortcut learning in deep neural networks (2020, background): “nding of shortcut learning which has led to a variety of mathematical concepts and an increasing amount of work under different terms such as learning under covariate shift [16], anti-causal learning [17], dataset bias [18], the tank legend [19] and the Clever Hans effect [20]. This perspective aims”
A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, background): “In particular, it is argued that disentanglement should lead to a better sample complexity of learning (Bengio et al., 2013; Schölkopf et al., 2012; Peters et al., 2017).”
Toward Causal Representation Learning (2021, methodology): “1) Changing (or performing an intervention upon) one mechanism P (Xi|PAi) does not change any of the other mechanisms P (Xj |PAj) (i = j) [220].”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “There has recently been a great deal of research and interest on causality in the machine learning community (Schölkopf et al. 2012, Guyon 2013, Mooij et al. 2016, Guyon 2014).”
Discovering Causal Signals in Images (2016, background): “In this example, ICM is satisfied under the correct causal direction X → Y , but violated under the wrong causal direction Y → X [3].”
Towards Causal Representation Learning (2021, background): “The SMS hypothesis was stated in [181, 24, 221, 115], and in earlier form in [218, 279, 220].”
Learning explanations that are hard to vary (2020, background): “ected to remain invariant under interventions or changing external conditions. This postulate has appeared in various forms in the literature [Haavelmo, 1943, Simon, 1953, Hurwicz, 1962, Pearl, 2009, Schölkopf et al., 2012].14 Based on this insight, it was proposed that regression based on causal fe”
Simpson's Paradox in COVID-19 Case Fatality Rates: A Mediation Analysis of Age-Related Causal Effects (2020, background): “Causal mechanisms, on the other hand, constitute stable (or invariant) units, which are often largely independent of other components of a system and should thus be transferable even if the distribution of some features changes [19], [57].”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “…a setting where few factors in a causal generative model change, which, by the independent causal mechanisms principle, leaves the others invariant (Schölkopf et al., 2012). representation learning is to learn a function r(x) mapping the observations to a low-dimensional vector that contains all…”
Causality for Machine Learning (2019, background): “ effect of interventions, it needs to be robust with respect to generalizing from an observational distribution to certain interventional distributions. One can express the above insights as follows (Schölkopf et al., 2012; Peters et al., 2017): Independent Causal Mechanisms (ICM) Principle. The cau”
Recurrent Independent Mechanisms (2019, background): “ables determined by such models (Pearl, 2009). It has been argued that the individual modules tend to remain robust or invariant even as other modules change, e.g., in the case of distribution shift (Schölkopf et al., 2012; Peters et al., 2017). One may hypothesize that if a brain is able to solve m”
Invariant Risk Minimization (2019, background): “Irma: It sounds reasonable! What about the case where P (Y e|Xe) changes? Does this happen in normal supervised learning? I remember attending a lecture by Professor Schölkopf [45, 25] where he mentioned that P (Y e|Xe) is often invariant across environments when X is a cause of Y , and that it ofte”
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “ generating X [39, 40]. The elementary ingredients G i;i= 1;:::;Kof the causal process work on their own and are changeable without affecting others, reﬂecting the independent 2 mechanisms assumption [43]. However, for some of the factors a hierarchical structure may exist for which this may only ho”
Generalization in anti-causal learning (2018, background): “The former refers to a situation where we are learning to predict effect from cause, and for the latter it is vice versa (Schölkopf et al., 2012).”