A General Metric for Riemannian Manifold Hamiltonian Monte Carlo

By Michael Betancourt. 2012

108 citations recorded by the source corpus.

Markov Chain Monte Carlo (MCMC) is an invaluable means of inference with complicated models, and Hamiltonian Monte Carlo, in particular Riemannian Manifold Hamiltonian Monte Carlo (RMHMC), has demonstrated success in many challenging problems. Current RMHMC implementations, however, rely on a Riemannian metric that limits their application. In this paper I propose a new metric for RMHMC without these limitations and verify its success on a distribution that emulates many hierarchical and latent models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability, Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

Generalizing Hamiltonian Monte Carlo with Neural Networks (2017, methodology): “It would be interesting to explore combining our work with the use of HMC in a minibatch setting (Chen et al., 2014); with shadow Hamiltonians (Izaguirre & Hampton, 2004); with gradient pre-conditioning approaches similar to those used in Riemannian HMC (Girolami et al., 2009; Betancourt, 2013); wit”
Stan: A Probabilistic Programming Language (2017, methodology): “…Monte Carlo (RHMC) introduces a location-dependent metric that can overcome these ﬁnal hurdles (Girolami and Calderhead 2011); Stan has a prototype implementation of RHMC based on the SoftAbs metric of (Betancourt 2012), using a generalization of NUTS to Riemannian manifolds (Betancourt 2013).”
A review of automatic differentiation and its efficient implementation (2018, methodology): “…Hamiltonian Monte Carlo (RHMC) is a Markov chain Monte Carlo sampler that requires the second and third order derivatives of the log posterior distribution, and can overcome severe geometrical pathologies ﬁrst-order sampling methods cannot (Girolami, Calderhead, & Chin, 2013; Betancourt, 2013).”
Expectation Propagation as a Way of Life (2020, background): “One idea is to first perform the subtraction in the second line above, then do an eigendecomposition, keeping the eigenvectors of (Σnew k ) −1 but taking any negative eigenvalues and replacing them with small positive numbers as in the SoftAbs map of Betancourt (2013). It is not clear whether some s”
Expectation Propagation as a Way of Life: A Framework for Bayesian Inference on Partitioned Data (2014, methodology): “One idea, as in the SoftAbs map of Betancourt (2013), is to do an eigendecomposition, keep the eigenvectors but replace all negative eigenvalues with a small positive number and reconstruct the matrix.”
The Geometric Foundations of Hamiltonian Monte Carlo (2014, methodology): “This motivates using the global covariance of the target distribution for Euclidean Hamiltonian Monte Carlo and the SoftAbs metric (Betancourt, 2013b) for Riemannian Hamiltonian Monte Carlo; for further discussion see Livingstone and Girolami (2014).”
Model Selection in Bayesian Neural Networks via Horseshoe Priors (2017, methodology): “We used the diagonal softabs metric proposed in (Betancourt, 2013).”
Bayesian computation: a summary of the current state, and samples backwards and forwards (2015, background): “A robustified version of such a metric has been introduced in Betancourt (2013) and termed”
Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, background): “dom walk Metropolis, in the sense that it is geometrically ergodic for targets with tail decay of expfj jgfor >1 (see Taylor (2014)). A robustied version of such a metric has been introduced in Betancourt (2013) and termed SoftAbs. Here one approximates the absolute value of the eigenspectrum of ”