Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations

By Linda S. L. Tan, David J. Nott. 2012

49 citations recorded by the source corpus.

The effects of different parametrizations on the convergence of Bayesian computational algorithms for hierarchical models are well explored. Techniques such as centering, noncentering and partial noncentering can be used to accelerate convergence in MCMC and EM algorithms but are still not well studied for variational Bayes (VB) methods. As a fast deterministic approach to posterior approximation, VB is attracting increasing interest due to its suitability for large high-dimensional data. Use of different parametrizations for VB has not only computational but also statistical implications, as different parametrizations are associated with different factorized posterior approximations. We examine the use of partially noncentered parametrizations in VB for generalized linear mixed models (GLMMs). Our paper makes four contributions. First, we show how to implement an algorithm called nonconjugate variational message passing for GLMMs. Second, we show that the partially noncentered parametrization can adapt to the quantity of information in the data and determine a parametrization close to optimal. Third, we show that partial noncentering can accelerate convergence and produce more accurate posterior approximations than centering or noncentering. Finally, we demonstrate how the variational lower bound, produced as part of the computation, can be useful for model selection.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, neutral): “Tan and Nott (2013, 2014) applied these message-passing methods to generalized linear mixed models (and also combined them with SVI).”
Variational inference for count response semiparametric regression (2013, unknown): “Recently Tan & Nott (2013) used an extension of MFVB, known as non-conjugate variational message passing, to handle Poisson mixed models for longitudinal data and their lead is followed here for more general classes of count response semiparametric regression models.”
A Stochastic Variational Framework for Fitting and Diagnosing Generalized Linear Mixed Models (2012, methodology): “ cases of the partially noncentered parametrization. Rewriting (2) as y i= Z iW i+ Z i~ i+ i; we can apply VB to the reparametrized model and assume that q(;~ 1;:::;~ n) = q() Q n i=1 q(~ i). Tan and Nott (2013) showed that the resulting VB algorithm converges in one iteration when W i= (ZT i Q iZ ”
Semiparametric Mean Field Variational Bayes: General Principles and Numerical Issues (2016, methodology): “…concerned with deterministic approximate Bayesian inference, such as Barber and Bishop (1997), Honkela et al. (2010), Knowles and Minka (2011), Tan and Nott (2013), Wand (2014) and Menictas and Wand (2015), have demonstrated that modification of mean field variational Bayes (e.g. Wainwright…”
Fully simplified multivariate normal updates in non-conjugate variational message passing (2014, unknown): “Details are given in Appendix B of Tan and Nott (2013).”
Laplace Variational Approximation for Semiparametric Regression in the Presence of Heteroskedastic Errors (2016, methodology): “Examples of such models include logistic regression (Jaakkola and Jordan 1997), nonparametric regression with measurement error (Pham, Ormerod, and Wand 2013), generalized linear mixed models (Tan and Nott 2013), and correlated topic models (Blei and Lafferty 2007).”
Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models (2016, background): “See Zhao et al. (2006), Tan and David (2013), and Yu, Zhang, and Yau (2014) for details on this reparameterization.”
Gaussian variational approximation with sparse precision matrices (2016, methodology): “ , and count responses, where y ij ˘Poisson( ij) with the log link function g( ij) = log( ij). Variational methods have been considered for ecient computation in GLMMs by Ormerod and Wand (2012); Tan and Nott (2013, 2014); Lee and Wand (2016a,b), among others. We parameterize the elements of the ran”