Semi-supervised learning, causality, and the conditional cluster assumption

By Julius von Kügelgen, Marco Loog, Alexander Mey, Bernhard Schölkopf. 2019

30 citations recorded by the source corpus.

While the success of semi-supervised learning (SSL) is still not fully understood, Scholkopf et al. (2012) have established a link to the principle of independent causal mechanisms. They conclude that SSL should be impossible when predicting a target variable from its causes, but possible when predicting it from its effects. Since both these cases are somewhat restrictive, we extend their work by considering classification using cause and effect features at the same time, such as predicting disease from both risk factors and symptoms. While standard SSL exploits information contained in the marginal distribution of all inputs (to improve the estimate of the conditional distribution of the target given inputs), we argue that in our more general setting we should use information in the conditional distribution of effect features given causal features. We explore how this insight generalises the previous understanding, and how it relates to and can be exploited algorithmically for SSL.

Subfields: Artificial Intelligence, Molecular Biology, Representation & Self-Supervised Learning, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoshua Bengio
2. Bernhard Schölkopf
3. Anirudh Goyal
4. Stefan Bauer
5. Nan Rosemary Ke
6. Francesco Locatello
7. Kyunghyun Cho
8. Quoc V. Le

In Molecular Biology

1. Nal Kalchbrenner
2. Stefan Bauer
3. Yoshua Bengio
4. Anirudh Goyal
5. Nan Rosemary Ke
6. Francesco Locatello
7. Bernhard Schölkopf
8. Stefano Ermon

In Representation & Self-Supervised Learning

1. Bernhard Schölkopf
2. Francesco Locatello
3. Anirudh Goyal
4. Stefan Bauer
5. Yoshua Bengio
6. Nan Rosemary Ke
7. Nal Kalchbrenner
8. J. Urgen Schmidhuber

In Causal Inference & Discovery

1. Yoshua Bengio
2. Anirudh Goyal
3. Stefan Bauer
4. Francesco Locatello
5. Nan Rosemary Ke
6. Nal Kalchbrenner
7. Bernhard Schölkopf
8. Dominik Janzing

Citations in the corpus

• Toward Causal Representation Learning (2021, background): “On the SSL side, subsequent developments include further theoretical analyses [125], [188, Section 5.1.2] and a form of conditional SSL [261].”
• Causality for Machine Learning (2019, background): “…2016; Pfister et al., 2018a; Heinze-Deml et al., 2017; Heinze-Deml and Meinshausen, 2017).16 On the SSL side, subsequent developments include further theoretical analyses (Janzing and Schölkopf, 2015; Peters et al., 2017, Section 5.1.2) and a form of conditional SSL (von Kügelgen et al., 2019).”