SCAN: Learning Hierarchical Compositional Visual Concepts

By Irina Higgins, Nicolas Sonnerat, Loic Matthey, Arka Pal, Christopher P. Burgess, Matko Bosnjak, Murray Shanahan, Matthew Botvinick, Demis Hassabis, Alexander Lerchner. 2017

134 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, methodology): “While prior work (Steenbrugge et al., 2018; LaversanneFinot et al., 2018; Nair et al., 2018; Higgins et al., 2017b, 2018b) successfully applied disentanglement methods such as β-VAE on a variety of downstream tasks, it is not clear to us that these approaches and trained models performed well…”
Environmental drivers of systematicity and generalization in a situated agent (2019, background): “, 2018), explicit latent variables (Higgins et al., 2017) or other neuro-symbolic hybrid methods (Mao et al.”
Learning World Graphs to Accelerate Hierarchical Reinforcement Learning (2019, neutral): “the need of a well-calibrated latent goal space (Higgins et al., 2017; Dwiel et al., 2019; Nachum et al., 2018a); Section 6 briefly speculates on extensions of our framework to other environments as future directions.”
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “sequence modeling, curiosity-based exploration or even domain adaptation in reinforcement learning [45, 25, 37, 16, 18, 29, 47].”
On the Fairness of Disentangled Representations (2019, methodology): “Weaker forms of supervision like relational information or additional assumptions on the effect of the factors of variation were also studied [39, 16, 47, 31, 91, 26, 19, 41, 93, 62, 53, 81, 8] and applied in the sequential data and reinforcement learning settings [88, 85, 57, 69, 37, 38].”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, background): “Disentangled representations have been found to be more sample-efficient [29], less sensitive to nuisance variables [55], and better in terms of (systematic) generalization [1, 16, 28, 35, 70].”
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations (2018, methodology): “While prior work (Steenbrugge et al., 2018; Laversanne-Finot et al., 2018; Nair et al., 2018; Higgins et al., 2017b; 2018) successfully applied disentanglement methods such as β-VAE on a variety of downstream tasks, it is not clear to us that these approaches and trained models performed well…”
Theory and Evaluation Metrics for Learning Disentangled Representations (2019, background): “Disentanglement learning holds the key for understanding the world from observations, transferring knowledge across different tasks and domains, generating novel designs, and learning compositional concepts [5, 17, 24, 32, 39].”
Towards a Definition of Disentangled Representations (2018, background): “For example, subsequent tasks may safely ignore certain parts of such a representation to form abstractions [41, 53], or use novel recombinations of values within the subspaces for improved generalisation [40].”
Measuring Compositional Generalization: A Comprehensive Method on Realistic Data (2019, background): “Similarly, we can learn a new object shape and then understand its compositions with previously learned colors or materials (Johnson et al., 2017; Higgins et al., 2018).”
Disentangling Factors of Variation Using Few Labels (2019, background): “[22] Irina Higgins, Nicolas Sonnerat, Loic Matthey, Arka Pal, Christopher P Burgess, Matko Bošnjak, Murray Shanahan, Matthew Botvinick, Demis Hassabis, and Alexander Lerchner.”
Learning by Abstraction: The Neural State Machine (2019, methodology): “Our model connects to multiple lines of research, including works about concept acquisition [36], visual abstractions [24, 81, 58, 57, 87], compositionality [14, 38], and neural computation [28, 62, 7].”
Unsupervised Model Selection for Variational Disentangled Representation Learning (2019, background): “…and recently a number of papers have demonstrated that models with disentangled representations show improvements in terms of these shortcomings (Higgins et al., 2017b; 2018b; Achille et al., 2018; Steenbrugge et al., 2018; Nair et al., 2018; Laversanne-Finot et al., 2018; van Steenkiste et…”
If deep learning is the answer, what is the question? (2020, background): “…growing emphasis on the virtues of model-based computation in neuroscience, machine learning researchers are building powerful deep generative models that are capable of disentangling the world into its latent factors, and recomposing these to construct realistic synthetic 3D images 70,107,108 .”
Locality and compositionality in zero-shot learning (2019, background): “…supervised label classiﬁer ( FC ) • Unsupervised / reconstruction based / generative : Variational auto-encoders( VAE , Kingma & Welling, 2013), β -VAE (Higgins et al., 2017a), Adversarial auto-encoders ( AAE , Makhzani et al., 2015), • Local self-supervision and variants : Augmented Multiscale…”
Disentangling and Learning Robust Representations with Natural Clustering (2019, background): “Disentangled representations, defined as those where each latent variable is responsible for only one generative factor in the data while being relatively invariant to other factors [1], allow for more easily interpretable models, [2], and facilitate generalization to previously unseen combinations ”
Is Independence all you need? On the Generalization of Representations Learned from Correlated Data (2020, background): “Due to the induced structure, disentangled representations promise generalization to unseen scenarios (Higgins et al., 2017b), increased interpretability (Adel et al., 2018; Higgins et al., 2018) and faster learning on downstream tasks (van Steenkiste et al., 2019; Locatello et al., 2019a).”