Nonparametric Variational Auto-Encoders for Hierarchical Representation Learning

By Prasoon Goyal, Zhiting Hu, Xiaodan Liang, Chenyu Wang, Eric P. Xing, Carnegie Mellon. 2017

123 citations recorded by the source corpus.

The recently developed variational autoencoders (VAEs) have proved to be an effective confluence of the rich representational power of neural networks with Bayesian methods. However, most work on VAEs use a rather simple prior over the latent variables such as standard normal distribution, thereby restricting its applications to relatively simple phenomena. In this work, we propose hierarchical non-parametric variational autoencoders, which combines tree-structured Bayesian nonparametric priors with VAEs, to enable infinite flexibility of the latent representation space. Both the neural parameters and Bayesian priors are learned jointly using tailored variational inference. The resulting model induces a hierarchical structure of latent semantic concepts underlying the data corpus, and infers accurate representations of data instances. We apply our model in video representation learning. Our method is able to discover highly interpretable activity hierarchies, and obtain improved clustering accuracy and generalization capacity based on the learned rich representations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

BooVAE: Boosting Approach for Continual Learning of VAE (2019, background): “…( x ) we are given a dataset { x n } Nn =1 and consider an empirical distribution 1 Optimal Prior and Aggregated Posterior Hoffman & Johnson (2016); Goyal et al. (2017) discuss the choice of the prior distribution over latent space π ( z ) and observe that default choice of the Gaussian prior…”
VAE with a VampPrior (2017, methodology): “In [30] a Dirichlet process prior using a stickbreaking process was proposed, while [14] proposed a nested Chinese Restaurant Process.”
LaDDer: Latent Data Distribution Modelling with a Generative Prior (2020, unknown): “Several works have considered using more flexible distributions than a unit Gaussian as priors, such as a stick-breaking prior [30], a Chinese Restaurant Process prior [13] or a Gaussian mixture (GM) prior [7, 18, 27, 38].”
Learning Hierarchical Priors in VAEs (2019, methodology): “Several works improve the VAE by learning more complex priors such as the stick-breaking prior (Nalisnick & Smyth, 2017), a nested Chinese Restaurant Process prior (Goyal et al., 2017), Gaussian mixture priors (Dilokthanakul et al., 2016), or autoregressive priors (Chen et al., 2016b).”
DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder (2018, background): “ate posterior distribution over the latent variables to a simple prior such as standard normal distribution, thereby restricting the generated responses to a relatively simple (e.g., unimodal) scope (Goyal et al., 2017). A number of studies have sought GAN-based approaches (Goodfellow et al., 2014; ”
Dual Motion GAN for Future-Flow Embedded Video Prediction (2017, background): “Existing works [37, 32, 12, 8] often learn a whole latent vector z for all objects in the scene.”
Semi-Amortized Variational Autoencoders (2018, background): “Note that unlike to sequential VAEs that employ different latent variables at each time step (Chung et al., 2015; Fraccaro et al., 2016; Krishnan et al., 2017; Serban et al., 2017; Goyal et al., 2017a), in this work we focus on modeling the entire sequence with a global latent variable.”
Every Smile is Unique: Landmark-Guided Diverse Smile Generation (2018, methodology): “The second phase consists on fine-tuning the VAE and training the first layer LSTM on the dataset of sequences of landmark images {yn}Nn=1 for 20 epochs.”
Deep Bayesian Unsupervised Lifelong Learning (2021, methodology): “The MNIST database of 70,000 handwritten digit images [35] is widely used to evaluate deep generative models [23, 25, 24, 27, 26] for representation learning and LL models in both supervised [14, 18, 19] and unsupervised learning contexts [22].”
Can deep learning beat numerical weather prediction? (2021, methodology): “Other approaches to introduce physical constraints into the latent space of DL models are the adversarial autoencoder by Makhzani et al. [130] and the non-parametric Bayesian method by Goyal et al. [131].”