Variational Bayesian Inference with Stochastic Search

By John William Paisley, David M. Blei, Michael I. Jordan. 2012

505 citations recorded by the source corpus.

Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using this factorized distribution. Often not all integrals are in closed form, which is typically handled by using a lower bound. We present an alternative algorithm based on stochastic optimization that allows for direct optimization of the variational lower bound. This method uses control variates to reduce the variance of the stochastic search gradient, in which existing lower bounds can play an important role. We demonstrate the approach on two non-conjugate models: logistic regression and an approximation to the HDP.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

Citations in the corpus

Auto-Encoding Variational Bayes (2013, background): “rectly inﬂuence the estimate through the samples z(l) ˘q ˚(zjx), and it is impossible to differentiate through this sampling process. Existing work on stochastic variational bayes provide workarounds [BJP12], but not a solution to this problem. 2.3 Our estimator of the lower bound Under certain mild”
Neural Variational Inference and Learning in Belief Networks (2014, methodology): “of Paisley et al. (2012)), baselines do not depend on the form of the model or of the variational distribution and thus are easier to use.”
Deep AutoRegressive Networks (2013, positive): “To combat the high variance, we introduce a baseline b, inspired by control variates (Paisley et al., 2012): ∇θE [f(Hi)] ≈ E [ ∇̂θF ] (19) ∇̂θF = ∇θ log q(hi)(f(hi)− b) (20) where ∇̂θF denotes our estimator.”
BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling (2019, methodology): “The introduction of stochastic backpropagation (Paisley et al., 2012; Hoffman et al., 2013) and the variational auto-encoder (VAE) (Kingma, 2013; Rezende et al.”
Improved Variational Inference with Inverse Autoregressive Flow (2016, methodology): “n experiments we demonstrate that autoregressive ﬂow leads to signiﬁcant performance gains when applied to variational autoencoders for natural images. 1 Introduction Stochastic variational inference Blei et al. (2012); Hoffman et al. (2013) is a method for scalable posterior inference with large da”
Categorical Reparameterization with Gumbel-Softmax (2016, neutral): “…gradient estimation has traditionally focused on either score function estimators augmented with Monte Carlo variance reduction techniques (Paisley et al., 2012; Mnih & Gregor, 2014; Gu et al., 2016; Gregor et al., 2013), or biased path derivative estimators for Bernoulli variables…”
Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic (2016, methodology): “By using the deterministic biased estimator as a particular form of control variate (Ross, 2006; Paisley et al., 2012) for the Monte Carlo policy gradient estimator, we can effectively use both types of gradient information to construct a new estimator that in practice exhibits improved sample…”
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (2016, methodology): “For models with discrete latent variables, the discontinuous nature of which makes reparameterization not useful, a number of score function estimators have been developed [31, 17, 33, 28, 42, 18], which differ primarily in the variance reduction techniques used.”
Black Box Variational Inference (2013, background): “The second is based on control variates (Ross, 2002; Paisley et al., 2012), using the gradient of the log probability of the variational distribution.”
Stochastic variational inference (2012, methodology): “In other work, we built on this algorithm to give scalable inference methods for Bayesian nonparametric models of topic correlation (Paisley et al., 2012b) and tree structures of topics (Paisley et al., 2012c).”
The price of debiasing automatic metrics in natural language evalaution (2018, methodology): “The technique has also been used in machine learning and reinforcement learning to lower variance estimates of gradients (Greensmith et al., 2004; Paisley et al., 2012; Ranganath et al., 2014).”
Variational Inference: A Review for Statisticians (2016, background): “This first appeared independently in several articles (Ji, Shen, and West 2010; Nott et al. 2012; Paisley, Blei, and Jordan 2012; Wingate and Weber 2013).”
MuProp: Unbiased Backpropagation for Stochastic Neural Networks (2015, methodology): “The derivation of MuProp in Section 3 uses a variance reduction technique known as control variates (Paisley et al., 2012).”
Concrete Dropout (2017, background): “Several estimators are available for us to do this: for example the score function estimator (also known as a likelihood ratio estimator and Reinforce [6, 12, 30, 35]), or the pathwise derivative estimator (this estimator is also referred to in the literature as the re-parametrisation trick, infinit”
Assumed Density Filtering Methods for Learning Bayesian Neural Networks (2016, background): “Although more accurate alternate bounds have been proposed (Khan 2012 where { z Unfortunately, this estimator is known to have high variance and further variance reduction techniques are required (Paisley, Blei, and Jordan 2012).”
Discrete Variational Autoencoders (2016, background): “Unfortunately, this naive estimate is impractically high-variance, leading to slow training and poor performance (Paisley et al., 2012).”
Sequential Neural Models with Stochastic Layers (2016, background): “All the intractable expectations in (4) would typically be approximated by sampling, using the reparameterization trick [19, 23] or control variates [22] to obtain low-variance estimators of its gradients.”
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression (2012, methodology): “While our method somewhat resembles performing stochastic gradient descent on the variational objective function in parameter space (Paisley et al. 2012; Nott et al. 2012), the linear regression view gives insights which allow a more computationally efficient approach.”