Hamiltonian Monte Carlo for Hierarchical Models

By Michael Betancourt, Mark A. Girolami. 2013

483 citations recorded by the source corpus.

Hierarchical modeling provides a framework for modeling the complex interactions typical of problems in applied statistics. By capturing these relationships, however, hierarchical models also introduce distinctive pathologies that quickly limit the efficiency of most common methods of in- ference. In this paper we explore the use of Hamiltonian Monte Carlo for hierarchical models and demonstrate how the algorithm can overcome those pathologies in practical applications.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/f7f38d7d679b1247a0620dfbc718ca5d9ff0f447

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Understanding the Impact of Microcredit Expansions: A Bayesian Hierarchical Analysis of 7 Randomised Experiments (2015, unknown): “I use a particular subset of MCMC methods called Hamiltonian Monte Carlo (HMC) methods throughout this paper, which are particularly suited to estimating hierarchical models (Betancourt and Girolami, 2013).”
Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors (2018, background): “Non-centered parameterizations are known to lead to simpler posterior geometries (Betancourt & Girolami, 2015).”
Visualization in Bayesian workflow (2017, methodology): “A random-walk Metropolis Markov kernel, for example, will get stuck in the neck of the funnel, rejecting a long sequence of proposals until a rare proposal directed towards the bulk of the funnel is proposed (Betancourt and Girolami, 2015; Betancourt, 2017; Stan Development Team, 2017b).”
Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC (with Discussion) (2019, background): “In such a case, differences in step-size adaptation can lead to chains that have different behavior in the neighborhood of the narrow part of the funnel (Betancourt and Girolami, 2019).”
Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors (2020, background): “Reparameterization may avoid posterior multimodality (Papaspiliopoulos et al., 2007; Johnson et al., 2012; Betancourt and Girolami, 2015; Gorinova et al., 2019), but this is not automated for general problems.”
Model Selection in Bayesian Neural Networks via Horseshoe Priors (2017, background): “Recent work [2, 12] suggests that the problem can be alleviated by adopting non-centered parameterizations.”
Bayesian Workflow (2020, background): “Applied statistics (not just Bayesian statistics) has become increasingly computational and algorithmic, and this has placed workflow at the center of statistical practice (see, for example, Grolemund and Wickham, 2017, Bryan, 2017, and Yu and Kumbier, 2020), as well as in application areas (for exa”
Fast Threshold Tests for Detecting Discrimination (2017, methodology): “This obstacle can be addressed with Hamiltonian Monte Carlo (HMC) methods (Betancourt and Girolami, 2015; Chen et al., 2014; Neal et al., 2011), which propose new samples by numerically integrating along the gradient of the log posterior, allowing for more efficient exploration.”
Stan: A Probabilistic Programming Language (2017, methodology): “Betancourt and Girolami (2013) provide a detailed analysis of the beneﬁts of centered vs. non-centered parameterizations of hierarchical models in Euclidean and Riemannian HMC, with the conclusion that non-centered are best when data are sparse and centered when the data strongly identiﬁes…”
A Conceptual Introduction to Hamiltonian Monte Carlo (2017, background): “Because such neighborhoods are characteristic of many important models, in particular hierarchical models (Betancourt and Girolami, 2015), these pathologies cannot be ignored in applied practice.”
Asymptotically exact inference in differentiable generative models (2016, background): “Whether it is necessarily helpful to remove dependencies in pu like this for the proposed method is an open question and will likely be model specific; it has previously been found that non-centred parameterisations can be beneficial when performing MCMC inference in hierarchical models when the uno”
Validating Bayesian Inference Algorithms with Simulation-Based Calibration (2018, methodology): “While some algorithms, such as Hamiltonian Monte Carlo (Neal et al., 2011; Betancourt and Girolami, 2013) provide diagnostics capable of identifying this problem, these diagnostics are not available for general MCMC algorithms.”
Optimizing The Integrator Step Size for Hamiltonian Monte Carlo (2014, background): “This more robust strategy has proven es- pecially effective for hierarchical models that are particularly sensitive to this pathology (Betancourt and Girolami, 2015).”
Pathfinder: Parallel quasi-Newton variational inference (2021, unknown): “The posteriordb package includes the eight schools model of Rubin (1981), in both a centered and non-centered parameterization; see Papaspiliopoulos et al. (2003); Stan Development Team (2021b); Betancourt and Girolami (2015) for more information on these parameterizations.”