Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness

By Raphael Suter, Ðorðe Miladinovic, Bernhard Schölkopf, Stefan Bauer. 2018

186 citations recorded by the source corpus.

The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks. While various approaches aiming towards this goal have been proposed in recent times, a commonly accepted definition and validation procedure is missing. We provide a causal perspective on representation learning which covers disentanglement and domain shift robustness as special cases. Our causal framework allows us to introduce a new metric for the quantitative evaluation of deep latent variable models. We show how this metric can be estimated from labeled observational data and further provide an efficient estimation algorithm that scales linearly in the dataset size.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Causal Inference & Discovery

Citations in the corpus

Toward Causal Representation Learning (2021, background): “Such a factorization may, thus, be called entangled, a term that has gained popularity in machine learning [24], [110], [158], [247].”
Towards Causal Representation Learning (2021, background): “Such a factorization may thus be called entangled, a term that has gained popularity in machine learning [23, 109, 158, 247].”
Causality for Machine Learning (2019, background): “Such a factorization may thus be called entangled, a term that has gained popularity in machine learning (Bengio et al., 2012; Locatello et al., 2018a; Suter et al., 2018).”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, methodology): “In this paper, we consider the setup of the recent unsupervised methods [27, 26, 48, 42, 9, 52, 71, 10].”
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations (2018, background): “…perform well on realistic settings most of the other works have considered some more or less explicit form of supervision (Reed et al., 2014; Zhu et al., 2014; Yang et al., 2015; Kulkarni et al., 2015; Cheung et al., 2015; Mathieu et al., 2016; Narayanaswamy et al., 2017; Suter et al., 2018).”
Interventional Few-Shot Learning (2020, background): “, Y → X , where the assumption is that labels Y are disentangled enough to be treated as Independent Mechanism (IM) [43, 55], which generates observed images X through Y → X .”
Compositionality and Generalization In Emergent Languages (2020, background): “We consequently take inspiration from work on disentanglement in representation learning (Suter et al., 2019) to craft measures that quantify whether an emergent language follows one of the proposed composition strategies.”
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models (2020, background): “In particular, [55, 64] demonstrated the importance and effectiveness of accounting for interventional perspectives.”
Generative causal explanations of black-box classifiers (2020, methodology): “also develop a disentanglement procedure from principles of causality [71], casting the disentanglement task as learning latent factors that correspond to parent-less causes in the generative structural causal model.”
Deconfounded Image Captioning: A Causal Retrospect (2020, background): “For example, disen-tangled causal mechanisms are exploited to design more robust disentanglement measurements [58], generate coun-terfactual samples [59], [60], and alleviate negative effects brought by different biases [61], [62], [63], [64], [65].”
Structural Autoencoders Improve Representations for Generation and Transfer (2020, background): “This embeds an SCM into a larger model whose inputs and outputs may be high-dimensional and unstructured (e.g., images) [18, 11].”
Disentangling User Interest and Conformity for Recommendation with Causal Embedding (2021, background): “Learning representations in which different semantics are disentangled is crucial for robust use of neural models [6, 32, 41, 44].”
Causal Intervention for Weakly-Supervised Semantic Segmentation (2020, background): “We believe that such anti-causal assumption only holds when the label is as simple as the disentangled causal mechanisms [35, 50] (e.”
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style (2021, unknown): “This is in line with a recently proposed [105] shift towards causal representation learning [40, 76, 85, 87, 106, 107, 112, 123, 127], motivated by the fact that many underlying variables of interest may not be independent but causally related to each other.”
Counterfactual Generative Networks (2021, background): “, on ImageNet), we can interpret the generation of an image as a causal process (Kocaoglu et al., 2018; Goyal et al., 2019; Suter et al., 2019).”
CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models (2020, methodology): “[30] used causality to explain disentangled latent representations.”