beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

By Irina Higgins, Loic Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner. 2016

6,058 citations recorded by the source corpus.

Subfields: Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

Shortcut learning in deep neural networks (2020, background): “r downstream tasks [140], let alone outof-distribution generalisation. Research on disentanglement addresses this shortcoming by learning generative models with well-structured latent representations [141]. The goal is to recover the true generating factors of the data distribution from observations”
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, methodology): “The depths are bucketed into size bins. dSprites/location (Matthey et al., 2017) The dSprites dataset was originally designed to asses disentanglement properties of unsupervised learning algorithms.”
A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, methodology): “2 BOTTLENECK CAPACITY Higgins et al. (2017a) propose the β-VAE, introducing a hyperparameter in front of the KL regularizer of vanilla VAEs.”
Zero-Shot Text-to-Image Generation (2021, background): “Note that the bound only holds for β = 1, while in practice we find it helpful to use larger values (Higgins et al., 2016).”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, background): “Unfortunately the metrics recently proposed for quantifying disentanglement [25, 30, 6, 18] require an encoder network that maps input images to latent codes.”
Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters (2018, methodology): “In particular, we refer to p as an encoding distribution rather than a prior, and moreover we will use a framework akin to the β -VAE (Higgins et al., 2017) which better reﬂects our goal of efﬁcient coding.”
Autoregressive Quantile Networks for Generative Modeling (2018, background): “, 2016a), and even been used for unsupervised learning of disentangled representations (Higgins et al., 2017; Chen et al., 2016).”
Vector-quantized Image Modeling with Improved VQGAN (2021, background): “Many image generation models (Goodfellow et al., 2014; Kingma & Welling, 2014; Radford et al., 2016; Donahue et al., 2017; Higgins et al., 2017) have been studied for their capabilities in representation learning.”
Dual Contradistinctive Generative Autoencoder (2020, positive): “Variational autoencoder (VAE) [37] points to an exciting direction of generative models by developing an Evidence Lower BOund (ELBO) objec-tive [25, 12].”
GAN Inversion: A Survey (2021, neutral): “(DBMs) [154], [155], [156], [157], Variational Autoencoders (VAEs) [158], [159], Deep Autoregressive Models (DARs) [160], [161], [162], [163], [164], and Normalizing Flow Models (NFMs) [165], [166], [167], [168].”
Learning deep representations by mutual information estimation and maximization (2018, methodology): “For our experiments, we compare DIM against various unsupervised methods: Variational AutoEncoders (VAE, Kingma & Welling, 2013), β-VAE (Higgins et al., 2016; Alemi et al., 2016), Adversarial AutoEncoders (AAE, Makhzani et al.”
Learning Latent Dynamics for Planning from Pixels (2018, methodology): “The lines show medians and the areas show percentiles 5 to 95 over 4 seeds and 10 rollouts. where we include weighting factors { β d } Dd =1 analogously to the β -VAE by Higgins et al. (2016).”
Multimodal Unsupervised Image-to-Image Translation (2018, background): “For example, InfoGAN [57] and β-VAE [58] have been proposed to learn disentangled representations without supervision.”
Variational Autoencoders for Collaborative Filtering (2018, background): “[17] proposed β-vae, which leads to the same objective as Eq.”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Representation Disentanglement [113, 279, 335, 336, 337, 338, 339, 340, 341, 342]”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, methodology): “Beta-VAE [65] is a tuning of the method that has been observed to disentangle factors remarkably well.”
Toward Causal Representation Learning (2021, background): “Much of the existing work on disentanglement [62], [110], [135], [157]–[159], [202], [256] focuses on independent factors of variation.”