Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations

By Diane Bouchacourt, Ryota Tomioka, Sebastian Nowozin. 2017

336 citations recorded by the source corpus.

We would like to learn a representation of the data that reflects the semantics behind a specific grouping of the data, where within a group the samples share a common factor of variation. For example, consider a set of face images grouped by identity. We wish to anchor the semantics of the grouping into a disentangled representation that we can exploit. However, existing deep probabilistic models often assume that the samples are independent and identically distributed, thereby disregard the grouping information. We present the Multi-Level Variational Autoencoder (ML-VAE), a new deep probabilistic model for learning a disentangled representation of grouped data. The ML-VAE separates the latent representation into semantically relevant parts by working both at the group level and the observation level, while retaining efficient test-time inference. We experimentally show that our model (i) learns a semantically meaningful disentanglement, (ii) enables control over the latent representation, and (iii) generalises to unseen groups.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, background): “…(i) disentanglement learning with interactions (Thomas et al., 2017), (ii) when weak forms of supervision like grouping information are available (Bouchacourt et al., 2018; Shu et al., 2020; Hosoya, 2019; Locatello et al., 2020a), or (iii) when temporal structure is available for the learning…”
Weakly-Supervised Disentanglement Without Compromises (2020, methodology): “The case k = 1 where we actually know which factor of variation is not shared was previously considered in (Bouchacourt et al., 2018; Hosoya, 2019; Shu et al., 2020).”
Weakly Supervised Disentanglement with Guarantees (2019, methodology): “While existing methods based on weakly-supervised learning demonstrate empirical gains, there is no existing formalism for describing the theoretical guarantees conferred by different forms of weak supervision (Kulkarni et al., 2015; Reed et al., 2015; Bouchacourt et al., 2018).”
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “Model Parameter Values β-VAE β [1, 2, 4, 6, 8] AnnealedVAE cmax [5, 10, 25, 50, 75] iteration threshold 100000 γ 1000 FactorVAE γ [10, 20, 30, 40, 50] DIP-VAE-I λod [1, 2, 5, 10, 20] λd 10λod DIP-VAE-II λod [1, 2, 5, 10, 20] λd λod β-TCVAE β [1, 2, 4, 6, 8] Table 2: Hyperparameters of different meth”
On the Fairness of Disentangled Representations (2019, methodology): “Weaker forms of supervision like relational information or additional assumptions on the effect of the factors of variation were also studied [39, 16, 47, 31, 91, 26, 19, 41, 93, 62, 53, 81, 8] and applied in the sequential data and reinforcement learning settings [88, 85, 57, 69, 37, 38].”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, neutral): “Due to the initial poor performance of purely unsupervised methods, the ﬁeld initially focused on semi-supervised [60, 11, 55, 56, 43, 45] and weakly supervised approaches [31, 12, 40, 21, 75, 20, 15, 35, 77, 52, 46, 62, 8].”
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations (2018, background): “…to some degree) seem to be for example (i) disentanglement learning with interactions (Thomas et al., 2017), (ii) when weak forms of supervision e.g. through grouping information are available (Bouchacourt et al., 2018), or (iii) when temporal structure is available for the learning problem.”
Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness (2018, background): “, 2018) or when some sort of supervision is available through groupings in the dataset according to generative factors (Bouchacourt et al., 2017).”
Conditional variance penalties and domain shift robustness (2017, background): “As in CoRe , Bouchacourt et al. (2017) exploit grouped observations.”
Uncertainty in Multitask Transfer Learning (2018, background): “More recently, Edwards and Storkey [10] and Bouchacourt et al. [6] explore hierarchical Bayesian inference with neural networks and evaluate on new tasks.”
Disentangled Sequential Autoencoder (2018, background): “(2017); Bouchacourt et al. (2017). These two types of approaches are often mixed together, e.”
Structured Disentangled Representations (2018, methodology): “In addition to the work of Kim and Mnih [2018] and Chen et al. [2018], our objective is also related to, and generalizes, a number of recently-proposed modifications to the VAE objective (see Table 1 for an overview). Zhao et al. [2017] considers an objective that eliminates the mutual information i”
Disentangling Factors of Variation Using Few Labels (2019, background): “Model Parameter Values β-VAE β [1, 2, 4, 6, 8, 16] S(2)/S β-VAE β [1, 2, 4, 6, 8, 16] γsup [1, 2, 4, 6, 8, 16] FactorVAE γ [10, 20, 30, 40, 50, 100] S(2)/S FactorVAE γ [10, 20, 30, 40, 50, 100] γsup [10, 20, 30, 40, 50, 100] DIP-VAE-I λod [1, 2, 5, 10, 20, 50] λd 10λod S(2)/S DIP-VAE-I λod [1, 2, 5,”
Inverse molecular design using machine learning: Generative models for matter engineering (2018, background): “By jointly training the VAE to reproduce molecules and properties, in a semi-supervised fashion,the latentmolecular spacereorganizesitself so that molecules with similar properties are close to each other.”
Guarding Against Adversarial Domain Shifts with Counterfactual Regularization (2017, background): “As in CoRe, Bouchacourt et al. (2017) exploit grouped observations.”
Group-based Learning of Disentangled Representations with Generalizability for Novel Contents (2019, methodology): “First of all, the most essential difference of MLVAE [Bouchacourt et al., 2018] from GVAE is the way inferring the content (equation 5), where the posterior distribution is estimated as the product of Gaussians for individual instances (“evidence accumulation”): qψ(z|x1, . . . , xK) = 1 Z K∏ k=1 N…”
Interventional Robustness of Deep Latent Variable Models (2018, background): “, 2018) or when some sort of supervision is available through groupings in the dataset according to generative factors (Bouchacourt et al., 2017).”
Neural Style Transfer: A Review (2017, methodology): “exploit annotated data to supervise the mapping between inputs and attributes [103].”
Dynamical Variational Autoencoders: A Comprehensive Review (2020, background): “The authors of (Bouchacourt et al., 2018) propose a hierarchical “multi-level” VAE with two latent vectors that are defined at different data scales: One latent vector encodes a common content for a group of data, and one other latent vector encodes the style of subgroups of data within a group.”
Learning to Manipulate Individual Objects in an Image (2020, background): “To encourage the alignment of the learned representation with the underlying generative factors, several constraints have been proposed to enforce the disentanglement of the latent codes [17, 5, 1, 30, 22, 9, 4, 18].”