A General Framework for the Parametrization of Hierarchical Models

By Omiros Papaspiliopoulos, Gareth O. Roberts, Martin Skold. 2007

349 citations recorded by the source corpus.

In this paper, we describe centering and noncentering methodology as complementary techniques for use in parametrization of broad classes of hierarchical models, with a view to the construction of effective MCMC algorithms for exploring posterior distributions from these models. We give a clear qualitative understanding as to when centering and noncentering work well, and introduce theory concerning the convergence time complexity of Gibbs samplers using centered and noncentered parametrizations. We give general recipes for the construction of noncentered parametrizations, including an auxiliary variable technique called the state-space expansion technique. We also describe partially noncentered methods, and demonstrate their use in constructing robust Gibbs sampler algorithms whose convergence properties are not overly sensitive to the data.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Auto-Encoding Variational Bayes (2013, background): “ked a reparameterization trick that is perhaps best known in literature for a different application , namely an efﬁcient Gibbs sampling technique going under the name of non-centered parameterization [PRS07] or ancillary augmentation (AA) [YM11]. The essential parameterization trick is quite simple.”
Bayesian Compression for Deep Learning (2017, background): “…this particular posterior parametrization the negative KL-divergence from the conditional prior p ( W | z ) to the posterior q φ ( W | z ) is independent of z : This independence can be better understood if we consider a non-centered parametrization of the prior [Papaspiliopoulos et al., 2007].”
SMC2: an efficient algorithm for sequential analysis of state space models (2011, background): “This is due to both high dependence between parameters and the latent process, which aﬀects Gibbs sampling strategies (Papaspiliopoulos et al., 2007), and the diﬃculty in designing eﬃcient simulation schemes for sampling from π T ( x 0: T | θ ).”
Monte Carlo Gradient Estimation in Machine Learning (2019, methodology): “This is a way to reparameterise a probabilistic system; it is often used in Monte Carlo methods, where it is referred to as the non-centred parameterisation (Papaspiliopoulos et al., 2007) or as a reparameterisation trick (Kingma and Welling, 2014b).”
Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors (2020, background): “Reparameterization may avoid posterior multimodality (Papaspiliopoulos et al., 2007; Johnson et al., 2012; Betancourt and Girolami, 2015; Gorinova et al., 2019), but this is not automated for general problems.”
Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, methodology): “While research on MCMC theory and methodology is still active and continually branching (Papaspiliopoulos et al. 2007; An-drieu and Roberts 2009; (cid:32)Latuszy´nski et al. 2011; Douc and Robert 2011), progressively incorporating the capacities of parallel processors and GPUs (Lee et al. 2009;…”
Bayesian Nonparametric Modelling of the Return Distribution with Stochastic Volatility (2011, positive): “the mixing have been proposed in the literature: reparametrize the model into a form that reduces the dependence between the parameters and the log-volatilities (Gelfand et al. 1995; Papaspiliopoulos et al. 2007) or to use a scheme that updates jointly the parameters and the log-volatilities (Kim et”
The correlated pseudomarginal method (2015, neutral): “Related ideas have been explored in (Papaspiliopoulos et al., 2007).”
Variational Inference for Sparse and Undirected Models (2016, background): “Many models can be rewritten in a noncentered form where the parameters and hyperparmeters are a priori independen (Papaspiliopoulos et al., 2007; Betancourt & Girolami, 2013).”
A Stochastic Variational Framework for Fitting and Diagnosing Generalized Linear Mixed Models (2012, background): “The SA steps in lines 10 and 11 were introduced by Robbins and Monro (1951) for optimizing an objective function which in our case is the lower bound L, with local variational parameters optimized as a function of the global ones.”
Validating Bayesian Inference Algorithms with Simulation-Based Calibration (2018, methodology): “Hierarchical models implemented with a centered parameterization (Papaspiliopoulos, Roberts and Sk¨old, 2007) are known to exhibit a challenging geometry that can cause MCMC algorithms to return biased posterior samples.”
To Center or Not to Center: That Is Not the Question—An Ancillarity–Sufficiency Interweaving Strategy (ASIS) for Boosting MCMC Efficiency (2011, methodology): “Section 2 defines the ASIS and more generally the component-wise ASIS, and uses examples from the works of Liu and Wu (1999) and Papaspiliopoulos, Roberts, and Skold (2003, 2007) to show how, why, and when ASIS works.”
Asymptotically exact inference in differentiable generative models (2016, background): “Whether it is necessarily helpful to remove dependencies in pu like this for the proposed method is an open question and will likely be model specific; it has previously been found that non-centred parameterisations can be beneficial when performing MCMC inference in hierarchical models when the uno”
Nonparametric estimation of diffusions: a differential equations approach (2012, background): “This type of implementation, which involves alternating between the X and Z variables, is typical of Markov chain Monte Carlo algorithms based on non-centered parameterisations, see Papaspiliopoulos et al. (2007) for details.”