Inadequacy of interval estimates corresponding to variational Bayesian approximations

By D. M. Titterington. 2005

128 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (2009, unknown): “First, even though VB methods seem often to approximate well the posterior mode (Wang and Titterington, 2006), the posterior variance can be (sometimes severely) underestimated; see Bishop (2006), chapter 10, and Wang and Titterington (2005).”
A Stochastic Variational Framework for Fitting and Diagnosing Generalized Linear Mixed Models (2012, background): “nd the impact of this restriction depends on how strong posterior dependence is among the factored variables. In VB, the posterior variance tends to be underestimated and this issue has been noted by Wang and Titterington (2005) and Bishop (2006). Recently, Zhao and Marriott (2013) proposed some dia”
Mean field variational bayes for elaborate distributions (2011, background): “In particular, the spreads of posterior densities are often under-approximated (e.g., Wang and Titterington 2005).”
Fast Bayesian Inference in Dirichlet Process Mixture Models (2011, background): “Wang and Titterington (2005) showed a tendency of VB to underestimate uncertainty in mixture models.”
Semiparametric Mean Field Variational Bayes: General Principles and Numerical Issues (2016, background): “Some examples, with explanations for the inaccuracy, are given in Wang and Titterington (2005) and Neville et al. (2014).”
Calibrating general posterior credible regions (2015, methodology): “ parameters of that family to minimize the Kullback{Leibler divergence from the true posterior. The variational posterior is simple by construction and, if carefully chosen, will be consistent (e.g., Wang and Titterington 2005), but as noted in Blei et al. (2016), misspecication causes the variation”
Boosting Variational Inference (2016, background): “But this assumption also means that MFVI eﬀectively cannot capture multimodality and that MFVI tends to underestimate the posterior covariance, sometimes drastically [Bishop, 2006, Wang and Titterington, 2004, Turner and Sahani, 2011, Rue et al., 2009, MacKay, 2003].”
Gaussian variational approximation with sparse precision matrices (2016, background): “Such independence assumptions have been shown to result in underestimation of the posterior variance (Wang and Titterington, 2005; Bishop, 2006).”
Regression Density Estimation With Variational Methods and Stochastic Approximation (2012, methodology): “For example, Wang and Titterington (2005) showed in the case of Gaussian mixtures that the covariance matrices from variational Bayes approximation are too small compared with those obtained by asymptotic arguments from maximum likelihood estimation.”
Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes (2015, methodology): “By generalizing linear response methods from statistical physics [9–12] to exponential family variational posteriors, we develop a methodology that augments MFVB to deliver accurate uncertainty estimates for model variables—both for individual variables and coherently across variables.”