Mean field variational bayes for elaborate distributions

By Matt P. Wand, John Thomas Ormerod, Simone A. Padoan, Rudolf Frühwirth. 2011

189 citations recorded by the source corpus.

We develop strategies for mean eld variational Bayes approximate inference for Bayesian hierarchical models containing elaborate distributions. We loosely dene elaborate distributions to be those having more complicated forms compared with common distributions such as those in the Normal and Gamma families. Examples are Asymmetric Laplace, Skew Normal and Generalized Ex- treme Value distributions. Such models suer from the diculty that the param- eter updates do not admit closed form solutions. We circumvent this problem through a combination of (a) specially tailored auxiliary variables, (b) univariate quadrature schemes and (c) nite mixture approximations of troublesome den-

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, methodology): “This perspective enabled the design of automated software for a large class of models (Wand et al. 2011; Minka et al. 2014).”
Semiparametric Mean Field Variational Bayes: General Principles and Numerical Issues (2016, methodology): “The intractable integral in fSNEx1 was approximated using a trapezoidal quadrature scheme similar to that described in Appendix B.2 of Wand et al. (2011).”
Mean field variational Bayes for continuous sparse signal shrinkage: Pitfalls and remedies (2014, unknown): “[34], Section 3) means that the lessons and methodology apply to other Bayesian models containing distributions of this type.”
Variational Inference for Heteroscedastic Semiparametric Regression (2015, background): “We call this the locality property of MFVB and it is described in section 3 of Wand et al. (2011).”
A variational Bayes approach to variable selection (2017, background): “The VB approach is summarized in many places including Bishop (2006); Rue et al. (2009); Ormerod and Wand (2010); Carbonetto and Stephens (2011); Faes et al. (2011); Wand et al. (2011); Murphy (2012); Pham et al. (2013); Luts and Ormerod (2014); Neville et al. (2014); You et al. (2014).”
Model Selection in Bayesian Neural Networks via Horseshoe Priors (2017, unknown): “Therefore, we use a more convenient auxiliary variable parameterization (Wand et al., 2011), a ∼ C(0, b)⇐⇒ a(2) | λ ∼ Inv-Gamma((1) 2 , 1 λ );λ ∼ Inv-Gamma((1) 2 , 1 b2 ), where v ∼ Inv-Gamma(a, b) is the Inverse Gamma distribution with density p(v) ∝ v−a−1exp{−b/v} for v > 0.”
Simple Marginally Noninformative Prior Distributions for Covariance Matrices (2013, methodology): “The proposed covariance matrix family involves a multivariate extension of Result 5 of Wand et al. (2011) which revealed that Half-t distributions arise as a scale mixture of Inverse-Gamma distributions.”
Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors (2018, methodology): “Instead of directly parameterizing the Half-Cauchy random variables in Equations 1 and 2, we use a convenient auxiliary variable parameterization (Wand et al., 2011) ∼ , where v ∼ Inv-Gamma ( a, b ) is the Inverse Gamma distribution with density p ( v ) ∝ v − a − 1 exp b/v for v > 0 .”
Variational inference for count response semiparametric regression (2013, unknown): “Extension to various other response distributions, such as t, Skew Normal and Generalized Extreme Value is relatively straightforward using approaches described in Wand et al. (2011).”