Fast Gradient-Based Inference with Continuous Latent Variable Models in Auxiliary Form

By Diederik P. Kingma. 2013

41 citations recorded by the source corpus.

We propose a technique for increasing the efficiency of gradient-based inference and learning in Bayesian networks with multiple layers of continuous latent vari- ables. We show that, in many cases, it is possible to express such models in an auxiliary form, where continuous latent variables are conditionally deterministic given their parents and a set of independent auxiliary variables. Variables of mod- els in this auxiliary form have much larger Markov blankets, leading to significant speedups in gradient-based inference, e.g. rapid mixing Hybrid Monte Carlo and efficient gradient-based optimization. The relative efficiency is confirmed in ex- periments.

Subfields: Artificial Intelligence, Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

NIPS 2016 Tutorial: Generative Adversarial Networks (2016, background): “Currently, the most popular approach to variational learning in deep generative models is the variational autoencoder (Kingma, 2013; Rezende et al., 2014) or VAE.”
Variational Dropout and the Local Reparameterization Trick (2015, background): “Variable reparameterizations have a long history in the statistics literature, but have only recently found use for efficient gradient-based machine learning and inference [4, 13, 19].”
Deep learning methods for forecasting COVID-19 time-Series data: A Comparative study (2020, unknown): “Variational autoencoders The variational autoencoders (VAE) belong to generative models that use learned approximate inference and could be constructed based on gradient-based techniques [41,42] .”