Variational Message Passing

By John M. Winn, Charles M. Bishop. 2005

833 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, methodology): “CAVI can also be seen as a “message passing” algorithm (Winn and Bishop 2005), iteratively updating each random vari-able’s variational parameters based on the variational parameters of the variables in its Markov blanket.”
Automatic Differentiation Variational Inference (2016, background): “Winn and Bishop (2005); Minka et al. (2014) extend this to graphical models with non-conjugate relationships by either using custom approximations or an expensive sampling approach.”
Automated Variational Inference in Probabilistic Programming (2013, background): “Variational inference [12–14] offers a powerful, deterministic approximation to exact Bayesian inference in complex distributions.”
Interpretable Neural Predictions with Differentiable Binary Variables (2019, methodology): “The rectiﬁed Gaussian distribution (Socci et al., 1998), in particular, has found applications to factor analysis (Harva and Kaban, 2005) and approximate inference in graphical models (Winn and Bishop, 2005).”
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression (2012, methodology): “e calculated analytically. We choose not to do this for our method to investigate how eective our MC estimation strategy can be. For completeness we also compare to variational message passing (VMP, Winn and Bishop, 2006), a message passing implementation of VBEM, and expectation propagation (EP, Mi”
Black-box $\alpha$-divergence Minimization (2015, background): “VMP stands for variational message passing described in Minka (2005), which should be distinguished from another version in Winn & Bishop (2005).”
SOLAR: Deep Structured Latent Representations for Model-Based Reinforcement Learning (2018, methodology): “Thus, the ELBO is given by Prior work has shown that, for conjugate exponential models such as the Bayesian LDS, the parameters can be updated using natural gradients, which can be computed in closed form using the variational message passing framework [44].”
Gaussian Process Regression Networks (2011, methodology): “Rasmussen and Williams (2006) and Bishop (2006) have a discussion about how to create and combine kernels.”
Latent-Space Variational Bayes (2008, methodology): “…not have the same asymptotic guarantees as MCMC. Recently, Variational Bayesian Expectation-Maximization (VBEM) [13], [14] has become one of the standard VB approximate inference methods and has been successfully applied to many interesting latent variable models [14], [15], [16], [17], [18], [19].”
Stochastic variational inference (2012, background): “It is the principal assumption made in many automated inference engines, such as VIBES (Bishop et al., 2003; Winn and Bishop, 2005).”
What, where and who? Classifying events by scene and object recognition (2007, methodology): “Here, we use the variational method based on Variational Message Passing [24] provided in [6] for an approximation.”
Stochastic Expectation Propagation (2015, background): “The same pathology exists for the broader class of power EP (PEP) algorithms [12] that includes variational message passing [13].”
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning (2018, unknown): “Johnson et al. (2016) build off of Hoffman et al. (2013) and Winn & Bishop (2005), who show that, for conjugate exponential models, the variational model parameters can be updated using natural gradients of the form ∇̃ωL = ω0 +BEq [tF,Σ(F,Σ)]− ω , (7) Where ω denotes the MNIW parameters of the…”
Power EP (2004, methodology): “This way of writing the mean-field updates is known as Variational Message Passing (Winn & Bishop, 2004).”
Automatic Variational Inference in Stan (2015, methodology): “NET, which implements variational message passing [15], an efficient algorithm for conditionally conjugate graphical models.”
Non-conjugate Variational Message Passing for Multinomial and Binary Regression (2011, background): “Variational Message Passing [20] is a message passing implementation of the mean-field approximation [1, 2], also known as variational Bayes (VB).”
Turing: A Language for Flexible Probabilistic Inference (2018, methodology): “Once the computation graph is constructed, a Gibbs sampler or message passing algorithm [Minka, 2001, Winn and Bishop, 2005] can Sampler Support discrete variables?”