Disentangling Disentanglement in Variational Autoencoders

By Emile Mathieu, Tom Rainforth, N. Siddharth, Yee Whye Teh. 2018

319 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

On Variational Bounds of Mutual Information (2019, background): “While the exact definition of disentangling remains elusive (Locatello et al., 2018; Higgins et al., 2018; Mathieu et al., 2018), many papers have focused on reducing statistical dependency between latent variables as a proxy (Kim & Mnih, 2018; Chen et al., 2018; Kumar et al., 2017).”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, neutral): “Whereas one could believe this distributional mismatch to be an undesired property, we recall the arguments conjectured in multiple previous works (Hoﬀman and Johnson, 2016; Burgess et al., 2017; Mathieu et al., 2019).”
Disentangling Factors of Variation Using Few Labels (2019, background): “Current state-of-the-art unsupervised disentanglement approaches enrich the Variational Autoencoder (VAE) [33] objective with different unsupervised regularizers that aim to encourage disentangled representations [20, 5, 31, 6, 36, 54, 44, 53].”
Unsupervised Model Selection for Variational Disentangled Representation Learning (2019, background): “…desirable for preserving information about the data that may be useful for subsequent tasks, in practice models often fail to discover and represent the full set of the data generative factors due to restrictions on both the observed data distribution and the model capacity (Mathieu et al., 2019).”
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding (2020, background): “Several recent studies have brought to light shortcomings in a number of proposed disentanglement metrics [32, 36, 37, 50, 65], many of which have been compiled in the DisLib benchmark.”
Biological underpinnings for lifelong learning machines (2022, background): “Several bio-inspired mechanisms contribute to flexible representations that facilitate task overlap and composition. and factorized representations found in autoencoders 165,166 , and context-dependent schemas 79,164 .”
Lifelong Mixture of Variational Autoencoders (2021, background): “A number of research works have been developed for capturing meaningful and disentangled data representations by using the VAE framework [29], [30], [31], [32], [33].”
CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models (2020, background): “On the contrary, by leveraging a few labels, one is able to recover the true model [21, 18].”
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models (2019, background): “This aspect is important from the perspective of downstream tasks, where better decomposed representations (Lipton, 2018; Mathieu et al., 2019) are more amenable for use on a wider variety of tasks.”
Controlling Text Edition by Changing Answers of Specific Questions (2021, background): “Implicit disentanglement (Higgins et al., 2017; Chen et al., 2018; Moyer et al., 2018; Mathieu et al., 2018; Kim and Mnih, 2018) separates the latent space into several components in a purely unsupervised way, expecting that each component corresponds to an attribute.”
Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition (2019, background): “[44]: ”The overlap factor is perhaps best understood by considering extremes: too little, and the latents effectively become a lookup table; too much, and the data and latents do not convey information about each other.”
Disentangling in Latent Space by Harnessing a Pretrained Generator (2020, background): “A properly disentangled representation can benefit semantic data mixing [30, 62], transfer learning for downstream tasks [6, 60], or even interpretability [43].”
Learning joint latent representations based on information maximization (2021, methodology): “Recently, various methods have been developed to induce disentangled representations without sacriﬁcing the reconstruction quality [20,21].”