Joint-VAE: Learning Disentangled Joint Continuous and Discrete Representations

By Emilien Dupont. 2018

275 citations recorded by the source corpus.

We present a framework for learning disentangled and interpretable jointly continuous and discrete representations in an unsupervised manner. By augmenting the continuous latent distribution of variational autoencoders with a relaxed discrete distribution and controlling the amount of information encoded in each latent unit, we show how continuous and categorical factors of variation can be discovered automatically from data. The learned model also contains an inference network which can infer quantities such as angle and width of objects from image data in a completely unsupervised manner. Our experiments show that the framework disentangles continuous and discrete generative factors on various datasets, including disentangling digit type from stroke thickness, angle and width on MNIST, chair type from azimuth and width on the Chairs dataset and age from azimuth on CelebA.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Learning latent representations across multiple data domains using Lifelong VAEGAN (2020, methodology): “arning. Many recent approaches would aim to modify the VAE framework in order to learn a meaningful representation of the data by imposing a large penalty on the Kullback-Leibler (KL) divergence term [7,17,25] or on the total correlation latent variables [12,20,28,30]. These approaches perform well ”
Lifelong Generative Modeling (2017, methodology): “8A similar idea is employed in work parallel to our own [21].”
Recent Advances in Autoencoder-Based Representation Learning (2018, methodology): “G/M VQ-VAE [20] (A) C /L [21] G JointVAE [23] C + N Variational Ladder AutoEncoders.”
Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN) (2020, background): “We observed some global features which are not usually seen in disentanglement experiments on MNIST (e.g. Chen et al., 2016; Dupont, 2018).”
Learning Disentangled Representations for Recommendation (2019, background): “Moreover, the controllability exhibited by many disentangled representations [19, 14, 10, 8, 9, 25] can potentially bring a new paradigm for recommendation, by giving users explicit control over the recommendation results and providing a more interactive experience.”
Learning joint latent representations based on information maximization (2021, methodology): “Dupont et al. [25], proposed to learn a joint distribution of continuous and discrete latent variables using VAEs in order to discover disentangled and interpretable representations.”
SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations (2020, methodology): “Following previous works (Dupont 2018), we used N = 1 and τ = 0.”
Learning Discrete and Continuous Factors of Data via Alternating Disentanglement (2019, background): “, 2017), JointVAE (Dupont, 2018), AAE with supervised discrete factors (Makhzani et al.”
Transferability and Hardness of Supervised Classification Tasks (2019, methodology): “The Celeb Faces Attributes (CelebA) set [34] was extensively used to evaluate transfer learning [17, 32, 33, 56].”
Disentanglement through Nonlinear ICA with General Incompressible-flow Networks (GIN) (2020, background): “We observed some global features which are not usually seen in disentanglement experiments on MNIST (e.g. Chen et al., 2016; Dupont, 2018).”
Disentangling and Learning Robust Representations with Natural Clustering (2019, methodology): “earning of disentangled latent representations. Similarly to our model, they predict a binary mask which they use to select which latent variables should feed into the decoder. Joint-VAE, proposed by Dupont (2018), uses a latent space that combines discrete and continuous latent variables. The Gumbe”
Disentangling User Interest and Conformity for Recommendation with Causal Embedding (2021, background): “Existing approaches mainly focus on computer vision [15, 16, 22, 23, 28].”
Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (2021, methodology): “Variational Autoencoder (VAE) [26] and Generative Adversarial Networks (GAN) [12], many disentanglement methods [6, 7, 11, 15, 17, 25, 32, 38, 48] have been proposed.”
Guided Variational Autoencoder for Disentanglement Learning (2020, methodology): “Latent Variables Traversal on MNIST: Comparison of traversal results from vanilla VAE [29], β-VAE [20], β-VAE with controlled capacity increase (CCβ-VAE), JointVAE [12] and our Guided-VAE on the MNIST dataset.”
Lifelong Mixture of Variational Autoencoders (2021, background): “A number of research works have been developed for capturing meaningful and disentangled data representations by using the VAE framework [29], [30], [31], [32], [33].”