Tutorial on Variational Autoencoders

By Carl Doersch. 2016

2,013 citations recorded by the source corpus.

In just three years, Variational Autoencoders (VAEs) have emerged as one of the most popular approaches to unsupervised learning of complicated distributions. VAEs are appealing because they are built on top of standard function approximators (neural networks), and can be trained with stochastic gradient descent. VAEs have already shown promise in generating many kinds of complicated data, including handwritten digits, faces, house numbers, CIFAR images, physical models of scenes, segmentation, and predicting the future from static images. This tutorial introduces the intuitions behind VAEs, explains the mathematics behind them, and describes some empirical behavior. No prior knowledge of variational Bayesian methods is assumed.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Generating Sentences by Editing Prototypes (2017, methodology): “in Equation 1) by only summing over x0that are lexically similar to x. 2.We lower bound the expectation over the edit prior (in Equation 2) using the evidence lower bound (ELBO) (Jordan et al., 1999; Doersch, 2016) which can be effectively approximated. We describe and motivate these approximations ”
Diagnosing and Enhancing VAE Models (2019, background): “Note that the only comparable prior work we are aware of related to the results in this section comes from Doersch (2016), where the implications of adopting Gaussian encoder/decoder pairs in the specialized case of r = d = 1 are brieﬂy considered.”
GAN Inversion: A Survey (2021, neutral): “Unlike other deep generative models (like Variational Autoencoders (VAEs) [179] or Generative Latent Optimization (GLO) [180]), the objective function”
LILA: Language-Informed Latent Actions (2021, methodology): “Instead, we adopt learned latent actions [5, 7, 8, 27, 28] a framework that uses conditional auto-encoders [29] to learn task-speciﬁc latent “action” spaces from demonstrations.”
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks (2016, methodology): “To further enforce the smoothness over the conditioning manifold and avoid overfitting [6, 14], we add the following regularization term to the objective of the generator during training,”
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks (2017, background): “avoid overfitting [9], [22], we add the following regularization term to the objective of the generator during training,”
Variational Autoencoders for Collaborative Filtering (2018, background): “Doersch [10] presents a review on different applications of vae to image data.”
A Point Set Generation Network for 3D Object Reconstruction from a Single Image (2016, background): “For more details about variational autoencoders, please refer to [6].”
Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, background): “having a fast training process through backpropagation [43].”
Generative replay with feedback connections as a general strategy for continual learning (2018, background): “Following [31], the output layer of the decoder network p ψ has a sigmoid non-linearity and the “reconstruction term” is given by the binary cross entropy between the original and decoded pixel values: whereby x i is the value of the i th pixel of the original input image x and ˜ x i is the value…”
An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders (2016, methodology): “Variational Autoencoders [9,10,11,22] modify this approach and make it tractable.”
Recent Advances in Autoencoder-Based Representation Learning (2018, background): “For a gentle introduction to VAEs we refer the reader to [31].”
Deep predictive policy training using reinforcement learning (2017, positive): “Variational autoencoders reproduce training data by sampling a latent variable (in our case the action a) from a prior distribution p(a), typically an isotropic Gaussian p(a) = N (a|0, I) [28], [29].”
Variational Prototype Replays for Continual Learning (2019, methodology): “Inspired by the design of variational autoencoders (Doersch, 2016), we propose variational encoders which learn a conditional class-representative Gaussian distribution with its mean µ nt,c and variance σ nt,c , given each input image I nt,c : ) .”
TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network (2017, background): “Variational Autoencoders (VAE) [3, 11], once trained, can be interpreted as generative models that produce samples from a distribution that approximates that of the training set.”
Time-Series Anomaly Detection Service at Microsoft (2019, methodology): “VAE was leveraged to model the reconstruction probabilities of normal time-series, while the abnormal points were reported if the reconstruction error was larger than a threshold.”