Generalizing Hamiltonian Monte Carlo with Neural Networks

By Daniel Lévy, Matthew D. Hoffman, Jascha Narain Sohl-Dickstein. 2017

136 citations recorded by the source corpus.

We present a general-purpose method to train Markov chain Monte Carlo kernels, parameterized by deep neural networks, that converge and mix quickly to their target distribution. Our method generalizes Hamiltonian Monte Carlo and is trained to maximize expected squared jumped distance, a proxy for mixing speed. We demonstrate large empirical gains on a collection of simple but challenging distributions, for instance achieving a 106x improvement in effective sample size in one case, and mixing when standard HMC makes no measurable progress in a second. Finally, we show quantitative and qualitative gains on a real-world task: latent-variable generative modeling. We release an open source TensorFlow implementation of the algorithm.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Stochastic Normalizing Flows (2020, background): “It has been demonstrated that combining learnable proposals/transformations with stochastic sampling techniques can improve expressiveness of the proposals (Salimans et al., 2015; Levy et al., 2017; Song et al., 2017; Hoffman et al., 2019; Hoffman, 2017).”
Denoising Diffusion Implicit Models (2020, methodology): “Our work is based on a large family of existing methods on learning generative models as transition operators of Markov chains (Sohl-Dickstein et al., 2015; Bengio et al., 2014; Salimans et al., 2014; Song et al., 2017; Goyal et al., 2017; Levy et al., 2017).”
Equivariant Hamiltonian Flows (2019, neutral): “The latter is the same interpretation as in HMC [22, 25, 21].”
Deep Energy Estimator Networks (2018, background): “But there are important new developments on the HMC front (Girolami & Calderhead, 2011; Levy et al., 2017), which DEEN can build on.”
NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport (2019, methodology): “More recently, Levy et al. (2018) proposed L2HMC, an adaptive MCMC algorithm to tune a generalization of the leapfrog integrator parameterized”
Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models (2020, background): “Salimans et al. (2015) on the other hand propose a hierarchical model inspired by the Hamiltonian dynamic, and Song et al. (2017); Levy et al. (2018) generalize Hamiltonian Monte Carlo (HMC) with trainable neural components.”
MintNet: Building Invertible Neural Networks with Masked Convolutions (2019, methodology): “They have been employed to investigate representations of deep classiﬁers [15], understand the cause of adversarial examples [14], learn transition operators for MCMC [28, 18], create generative models that are directly trainable by maximum likelihood [6, 5, 24, 16, 9, 1], and perform approximate in”
Practical tradeoffs between memory, compute, and performance in learned optimizers (2022, background): “…adversarial training (Xiong and Hsieh, 2020), few-shot learning (Ravi and Larochelle, 2016), swarm optimization (Cao et al., 2019), unsupervised learning (Metz et al., 2018), black box optimization (Chen et al., 2016), and MCMC sampling (Levy et al., 2017; Wang et al., 2017; Gong et al., 2018).”
Hamiltonian Generative Networks (2019, methodology): “We choose to use the trick proposed in the Hamiltonian Monte Carlo (HMC) literature (Neal et al., 2011; Salimans et al., 2015; Levy et al., 2017), which treats the momentum p as a latent variable (see Fig.”
Learning Neural Random Fields with Inclusive Auxiliary Generators (2018, background): “example, it is recently studied in [26] to learn MCMC transition kernels, also parameterized by neural networks, to improve the”
TensorFlow Eager: A Multi-Stage, Python-Embedded DSL for Machine Learning (2019, methodology): “In Figure 4 we show performance of an L2HMC (Levy et al., 2018) Eager, TensorFlow Eager with function , and TensorFlow on synthetic data running on the CPU.”
Exponential Family Estimation via Adversarial Dynamics Embedding (2019, methodology): “The proposed algorithm shares some similarities with recent advances in meta learning for sampling (Levy et al., 2018; Feng et al., 2017; Song et al., 2017; Gong et al., 2019), in which the sampler is parametrized via neural network and will be learned through certain objectives.”
Neural Network Renormalization Group (2018, background): “Further acceleration of the HMC using neural networks is an active research direction in deep learning [79, 80].”