Are Disentangled Representations Helpful for Abstract Visual Reasoning?

By Sjoerd van Steenkiste, Francesco Locatello, J. Urgen Schmidhuber, Olivier Bachem. 2019

219 citations recorded by the source corpus.

A disentangled representation encodes information about the salient factors of variation in the data independently. Although it is often argued that this representational format is useful in learning to solve many real-world down-stream tasks, there is little empirical evidence that supports this claim. In this paper, we conduct a large-scale study that investigates whether disentangled representations are more suitable for abstract reasoning tasks. Using two new tasks similar to Raven's Progressive Matrices, we evaluate the usefulness of the representations learned by 360 state-of-the-art unsupervised disentanglement models. Based on these representations, we train 3600 abstract reasoning models and observe that disentangled representations do in fact lead to better down-stream performance. In particular, they enable quicker learning using fewer samples.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

In Safety, Robustness & Interpretability

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, background): “For example, the applications in fairness (Locatello et al., 2019), abstract reasoning (van Steenkiste et al., 2019) and strong generalization (Locatello et al., 2020a) all conceptually rely on the disentanglement notion of (Eastwood and Williams, 2018).”
Toward Causal Representation Learning (2021, background): “Much of the existing work on disentanglement [62], [110], [135], [157]–[159], [202], [256] focuses on independent factors of variation.”
Towards Causal Representation Learning (2021, background): “Much of the existing work on disentanglement [109, 158, 159, 256, 157, 135, 202, 61] focuses on independent factors”
On the Binding Problem in Artificial Neural Networks (2020, background): “** A preliminary version of this work was presented at an ICML Workshop (van Steenkiste et al., 2019a).”
On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Disentangled representations hold the promise of generalization to unseen scenarios (Higgins et al., 2017b), increased interpretability (Adel et al., 2018; Higgins et al., 2018) and faster learning on downstream tasks (van Steenkiste et al., 2019; Locatello et al., 2019a).”
Weakly-Supervised Disentanglement Without Compromises (2020, methodology): “Further, we show that these representations are useful on a diverse suite of downstream tasks, including a novel experiment targeting strong generalization under covariate shifts, fairness (Locatello et al., 2019a) and abstract visual reasoning (van Steenkiste et al., 2019).”
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “Learning well-disentangled representations of complex sensory data has been identified as one of the key challenges in the quest for AI [1, 40, 26, 2, 42, 24, 47], since they should contain all the information present in the observations in a compact and interpretable structure [1, 21, 6] while bein”
On the Fairness of Disentangled Representations (2019, background): “Indeed, isolating each independent factor of variation into the independent components of a representation vector should make it both interpretable and simplify downstream prediction tasks [6, 7, 29, 35, 56, 58, 60, 74, 82, 87, 89, 90, 1, 28].”
GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations (2019, background): “[27]) combined with the ability to decompose scenes enables GENESIS to deliberately manipulate selected parts of the scene while leaving the rest unchanged.”
Unsupervised Model Selection for Variational Disentangled Representation Learning (2019, background): “…have demonstrated that models with disentangled representations show improvements in terms of these shortcomings (Higgins et al., 2017b; 2018b; Achille et al., 2018; Steenbrugge et al., 2018; Nair et al., 2018; Laversanne-Finot et al., 2018; van Steenkiste et al., 2019; Locatello et al., 2019).”
Better Set Representations For Relational Reasoning (2020, methodology): “Furthermore, we hypothesize that proper decomposition obtained via SRNs can improve both the accuracy and robustness of downstream reasoning modules, in line with prior work which suggests that disentanglement is desirable for downstream tasks [17].”
Set-Structured Latent Representations (2020, background): “Proposals for moving towards System 2 reasoning in learning systems involve creating new abilities for composition, combinatorial generalization, and disentanglement (Battaglia et al., 2018; van Steenkiste et al., 2019; Marra et al., 2019).”
Is Independence all you need? On the Generalization of Representations Learned from Correlated Data (2020, background): “Due to the induced structure, disentangled representations promise generalization to unseen scenarios (Higgins et al., 2017b), increased interpretability (Adel et al., 2018; Higgins et al., 2018) and faster learning on downstream tasks (van Steenkiste et al., 2019; Locatello et al., 2019a).”
Structural Autoencoders Improve Representations for Generation and Transfer (2020, methodology): “…compare the performance of each of the methods using a variety of different metrics to elucidate the quality of generated samples (using our "hybrid" sampling method as well as the standard "prior" sampling): disentanglement [29, 30], fairness [31], and abstract visual reasoning capabilities [32].”
Stratified Rule-Aware Network for Abstract Visual Reasoning (2020, unknown): “…reasoning on RPM test (Hoshen and Werman 2017; Barrett et al. 2018; Steenbrugge et al. 2018; Zhang et al. 2019a,b; Zheng, Zha, and Wei 2019; van Steenkiste et al. 2019; Wang, Jamnik, and Lio 2020), inspired by the progress of computer vision tasks (Krizhevsky, Sutskever, and Hinton 2012;…”