Optimizing The Integrator Step Size for Hamiltonian Monte Carlo

By Michael Betancourt, Simon Byrne, Mark A. Girolami. 2014

83 citations recorded by the source corpus.

Hamiltonian Monte Carlo can provide powerful inference in complex statistical problems, but ultimately its performance is sensitive to various tuning parameters. In this paper we use the underlying geometry of Hamiltonian Monte Carlo to construct a universal optimization criteria for tuning the step size of the symplectic integrator crucial to any implementation of the algorithm as well as diagnostics to monitor for any signs of invalidity. An immediate outcome of this result is that the suggested target average acceptance probability of 0.651 can be relaxed to $0.6 \lesssim a \lesssim 0.9$ with larger values more robust in practice.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families (2015, background): “Stochastic gradient based HMC methods, however, often suffer from low acceptance rates or additional bias that is hard to quantify [9].”
Exponential Integration for Hamiltonian Monte Carlo (2015, methodology): “9] for each combination of dataset and σ, as suggested by Betancourt et al. (2014a). The same (h, L) is used for expHMC, êxpHMC, and rmExpHMC, along with (2h, L/2) and (4h, L/4) to test larger steps.”
Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, methodology): “For a complete account of this important ﬂavour of MCMC, the reader is referred to Neal (2013), which inspired the description below; see also Betancourt et al. 2014 for a highly mathematical diﬀerential-geometry approach to HMC.”
A Conceptual Introduction to Hamiltonian Monte Carlo (2017, methodology): “Instead of relying on fragile heuristics, the method is built upon a rich theoretical foundation that makes it uniquely suited to the highdimensional problems of applied interest (Betancourt et al., 2014).”
Asymptotically exact inference in differentiable generative models (2016, background): “9 [13] and to minimise the occurrence of any rejections due to non-reversible geodesic steps or convergence failure in the iterative solver.”
scCODA is a Bayesian model for compositional single-cell data analysis (2021, methodology): “Parameter inference is performed via Hamiltonian Monte Carlo (HMC) sampling using ten leapfrog steps per iteration with automatic step size adjustment according to Betancourt et al. 32 .”
Bayesian computation: a summary of the current state, and samples backwards and forwards (2015, background): “What drives the exploration of the different values of H (θ, p ) is therefore the simulation of the momentum, which makes its calibration both quite inﬂuential and delicate (Betancourt et al. 2014) as it depends on the unknown normalising constant of the target.”