Propagation Algorithms for Variational Bayesian Learning

By Zoubin Ghahramani, Matthew J. Beal. 2000

365 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Composing graphical models with neural networks for structured representations and fast inference (2016, methodology): “First, the fact that there are only a fixed number of hidden units makes it unlikely that the model can [1,2,3] Pr bab listic graphical m els? [4,5,6] [4] Fox, Sudderth, Jord n, illsky.”
Stochastic Gradient Descent as Approximate Bayesian Inference (2017, background): “For the class of models where the conditional distributions are in the exponential family (CCEF class), the variational objective can be optimized by closed-form updates (Ghahramani and Beal, 2000), but this is a restricted class of models with conjugate priors.”
Hierarchical Variational Models (2015, background): “This leads to a computationally efficient optimization problem that can be solved (up to a local optimum) with coordinate descent (Bishop, 2006; Ghahramani and Beal, 2001) or gradient-based methods (Hoffman et al.”
The Generalized Reparameterization Gradient (2016, neutral): “For a certain class of probabilistic models, those where each conditional distribution is in an exponential family, we can easily use coordinate ascent optimization to minimize the KL divergence (Ghahramani and Beal, 2001).”
Deep Exponential Families (2014, background): “Unfortunately DEFs are a nonconjugate model—their local conditional distributions are difficult to compute—and so they are not amenable to simple inference algorithms like Gibbs sampling [9] or variational message passing [10].”
Variational inference for Dirichlet process mixtures (2006, background): “In this setting it is natural to consider the following family of distributions as meanfield variational approximations (Ghahramani and Beal 2001): q ν (w) = M i=1 exp{ν T i w i − a(w i )}, (14) where ν = {ν 1 , ν 2 , . . . , ν M } are variational parameters.”
Fast Variational Inference in the Conjugate Exponential Family (2012, background): “Variational bounds provide a convenient approach to approximate inference in a range of intractable models [Ghahramani and Beal, 2001].”
Latent-Space Variational Bayes (2008, unknown): “…criterion, but does not have the same asymptotic guarantees as MCMC. Recently, Variational Bayesian Expectation-Maximization (VBEM) [13], [14] has become one of the standard VB approximate inference methods and has been successfully applied to many interesting latent variable models…”
Sandwiching the marginal likelihood using bidirectional Monte Carlo (2015, methodology): “Variational Bayes (Hinton and van Camp, 1993; Waterhouse et al., 1996; Attias, 2000; Ghahramani and Beal, 2001) is an alternative set of techniques based on optimization.”
Graphical Models, Exponential Families, and Variational Inference (2008, methodology): “In the literature on the topic, however, the term “variational Bayes” has been reserved thus far for the application of the mean-field variational method to Bayesian inference [14, 88].”
Scalable Recommendation with Poisson Factorization (2013, background): “These auxiliary variables facilitate derivation and description of the algorithm [10, 14].”
Natural Conjugate Gradient in Variational Inference (2007, methodology): “This is equivalent to minimizing the Kullback–Leibler divergence DKL(q‖p) between q and p (Ghahramani and Beal, 2001).”
Adaptive BCI based on variational Bayesian Kalman filtering: an empirical evaluation (2004, methodology): “Contrary to previous use of variational methods for general state space models [34], [35], our approach tracks the parameters of a classifier and variational learning infers the model parameters and the precision of the process noise .”
Variational Message Passing (2005, methodology): “This approximation has been successfully used in many applications of variational methods (Attias, 2000, Ghahramani and Beal, 2001, Bishop, 1999).”
Applying Discrete PCA in Data Analysis (2004, methodology): “Note using this method, we have observed that the approximate Dirichlet posteriors estimated by the mean field method can have a posterior variance over 100 times smaller than the real posterior variance–in use we have observed this translates to very poor performance.”
Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations (2012, methodology): “Variational message passing (Winn and Bishop (2005)) is an algorithmic implementation of VB that can be applied to a general class of conjugate-exponential models (Attias ( 2000); Ghahramani and Beal (2001)).”
The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures (2003, methodology): “Readers familiar with the EM algorithm (Dempster et al. , 1977) may note the similarity between this iterative algorithm and EM. We call this procedure the Variational Bayesian EM Algorithm for reasons which will become clearer in the following sections; see also Attias (2000) and Ghahramani and Be”