Stochastic Gradient VB and the Variational Auto-Encoder

By Diederik P. Kingma, Max Welling. 2013

347 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images (2020, background): “Many types of generative models have ﬂourished in recent years, including likelihood-based generative models, which include autoregressive models (Uria et al., 2013), variational autoencoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014), and invertible ﬂows (Dinh et al., 2014; 2016).”
Variational Continual Learning (2017, methodology): “In this section we extend the VCL framework to encompass variational auto-encoders (VAEs) (Kingma and Welling, 2014; Rezende et al., 2014), a form of DGMs.”
MAVEN: Multi-Agent Variational Exploration (2019, methodology): “To address the intractability introduced by the use of continuous latent variables, we propose the use of state-of-the-art methods from variational inference [24, 42, 41, 25].”
Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning (2015, methodology): “x B.1. 4.1 Variational Information Lower Bound The MI can be made more tractable by deriving a lower bound to it and maximising this instead — here we present the bound derived by Barber & Agakov [1]. Using the entropy formulation of the MI (3) reveals that bounding the conditional entropy compo”
Generating Natural Adversarial Examples (2017, methodology): “…(Arora & Zhang, 2017), it is worth noting that we can also incorporate other generative models such as Variational Auto-Encoders (VAEs) (Kingma & Welling, 2014) into our framework, as used in Hu et al. (2017) to generate text with controllable attributes, whic we will explore in the…”
Iterative Amortized Inference (2018, methodology): “…form: While inference models have a long history, e.g. (Dayan et al., 1995), the most notable recent example is the variational auto-encoder (VAE) (Kingma & Welling, 2014; Rezende et al., 2014), which employs the reparameterization trick to propagate stochastic gradients from the generative model…”
Probabilistic Meta-Representations Of Neural Networks (2018, methodology): “aximizing the evidence lower bound (ELBO): logP(D;˘) EQ(Z) h log P(DjZ;˘)P(Z) Q(Z;) i (2) with logP(DjZ) ˇ1 S PS s=1 P(yjx;Ws;m) and Ws;m ˘P(WjZm). In practice, we apply the reparametrization trick [Kingma and Welling, 2014, Rezende et al., 2014, Titsias and Lázaro-Gredilla, 2015] and its variants [”
Model Selection in Bayesian Neural Networks via Horseshoe Priors (2017, background): “Recent progress in black box variational inference (Kingma and Welling, 2014; Rezende et al., 2014; Ranganath et al., 2014; Titsias and Lázaro-gredilla, 2014) subverts this difficulty.”
Gandiva: Introspective Cluster Scheduling for Deep Learning (2018, background): “GPU Time Packing Packing Job Util Slicing Max Gain (%) (mb/s) (mb/s) (%) VAE [29] 8.”
Learning Semantic Deformation Flows with 3D Convolutional Networks (2016, methodology): “Similar to the variational autoencoder [20], Kulkarni et al.”
An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos (2018, methodology): “It is still left as a future study to understand clearly, why the performance of a stochastic autoencoder such as VAE is better.”
Unsupervised Video Summarization with Adversarial LSTM Networks (2017, methodology): “6, we update the parameters θs, θe, θd and θc using the Stochastic Gradient Variational Bayes estimation [17, 16], adapted for recurrent networks”
Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials (2020, methodology): “In practice, we use stochastic variational inference (Ranganath, Gerrish, and Blei 2013; Kingma and Welling 2014) and subsample in order to perform inference and learning via gradient descent, yielding the loss function:”
Advances in Variational Inference (2017, methodology): “Stochastic gradient methods have been adapted to various settings, such gamma processes [82] and variational autoencoders [79].”