Stochastic Gradient Hamiltonian Monte Carlo

By Tianqi Chen, Emily B. Fox, Carlos Guestrin. 2014

1,015 citations recorded by the source corpus.

Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for defining distant proposals with high acceptance probabilities in a Metropolis-Hastings framework, enabling more efficient exploration of the state space than standard random-walk proposals. The popularity of such methods has grown significantly in recent years. However, a limitation of HMC methods is the required gradient computation for simulation of the Hamiltonian dynamical system--such computation is infeasible in problems involving a large sample size or streaming data. Instead, we must rely on a noisy gradient estimate computed from a subset of the data. In this paper, we explore the properties of such a stochastic gradient HMC approach. Surprisingly, the natural implementation of the stochastic approximation can be arbitrarily bad. To address this problem we introduce a variant that uses second-order Langevin dynamics with a friction term that counteracts the effects of the noisy gradient, maintaining the desired target distribution as the invariant distribution. Results on simulated data validate our theory. We also provide an application of our methods to a classification task using neural networks and to online Bayesian matrix factorization.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Generative Modeling by Estimating Gradients of the Data Distribution (2019, background): “(4), but it can often be ignored in practice [9, 12, 39].”
Generalizing Hamiltonian Monte Carlo with Neural Networks (2017, methodology): “It would be interesting to explore combining our work with the use of HMC in a minibatch setting (Chen et al., 2014); with shadow Hamiltonians (Izaguirre & Hampton, 2004); with gradient pre-conditioning approaches similar to those used in Riemannian HMC (Girolami et al., 2009; Betancourt, 2013);…”
Evidential Deep Learning to Quantify Classification Uncertainty (2018, methodology): “Improvement of the approximation techniques, such as Variational Bayes (VB) [2, 6, 27, 23, 9] and Stochastic Gradient Hamiltonian Monte Carlo (SG-HMC) [3] tailored specifically for scalable inference of BNNs is an active research field.”
BOHB: Robust and Efficient Hyperparameter Optimization at Scale (2018, methodology): “We used stochastic gradient Hamiltonian Monte-Carlo sampling (SGHMC) (Chen et al., 2014) with scale adaption (Springenberg et al., 2016) to sample the parameter vector of the network.”
A Simple Baseline for Bayesian Uncertainty in Deep Learning (2019, methodology): “3 We note that stochastic gradient Monte Carlo methods [5, 46] also use the SGD trajectory to construct samples from the approximate posterior.”
Entropy-SGD: biasing gradient descent into wide valleys (2016, methodology): “( However, if the learning rate η t → 0, we can also forgo the MH step Chen et al., 2014).”
Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families (2015, background): “There have been efforts to mimic HMC’s behaviour using stochastic gradients from mini-batches in Big Data [7], or stochastic ﬁnite differences in ABC [8].”
Improving Task-free Continual Learning by Distributionally Robust Memory Evolution (2022, background): “This WGF corresponds to Hamiltonian dynamics (Chen et al., 2014) and can be solved by the following evolution: { xt+1 − xt = vt, vt+1 − vt = −α∇U(x,θ)− τv + √ 2ταξt, (17)”
Bayesian dark knowledge (2015, neutral): “ an online fashion, just as SGD updates a point estimate of the parameters online. Furthermore, various extensions of SGLD have been proposed, including stochastic gradient hybrid Monte Carlo (SGHMC) [CFG14], stochastic gradient Nos´e-Hoover Thermostat (SG-NHT) [DFB +14] (which improves upon SGHMC),”
Stochastic Gradient Descent as Approximate Bayesian Inference (2017, methodology): “Other research suggests improvements to this issue, using Hamiltonian Monte-Carlo (Chen et al., 2014) or thermostats (Ding et al., 2014).”
Three Factors Influencing Minima in SGD (2017, methodology): “tice. 2 RELATED WORK The relationship between SGD and sampling a posterior distribution via stochastic Langevin methods has been the subject of discussion in a number of papers (Chaudhari et al.,2017;Chen et al., 2014;Ding et al.,2014;Vollmer et al.,2015;Welling & Teh,2011;Shang et al.,2015;Sato”
Learning Gradient Fields for Shape Generation (2020, neutral): “Even when such conditions fail to hold, the error in Equation 8 is usually negligible when α is small and T is large [54,10,14,42].”
Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo (2015, methodology): “These include Stochastic Gradient Langevin dynamics (SGLD) (Welling & Teh, 2011), Stochstic Gradient Fisher scoring (SGFS) (Ahn et al., 2012), Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) (Chen et al., 2014) as well as more recent Stochastic Gradient Nos´e-Hoover Thermostat (SGNHT) (Ding et a”
A Variational Analysis of Stochastic Gradient Algorithms (2016, methodology): “Other research suggests improvements to this issue, using Hamiltonian Monte-Carlo (Chen et al., 2014) or thermostats (Ding et al., 2014).”
RoBO : A Flexible and Robust Bayesian Optimization Framework in Python (2017, methodology): “A recent popular class of models are Bayesian neural networks [Neal, 1996], which combine well-calibrated uncertainty estimates with strong scalability to high dimensions and many data points; in particular, Snoek et al. [2015] used Bayesian linear regression based on the features in the last layer ”
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision (2019, methodology): “To tackle this problem, a wide variety of approximations have been explored [38, 22, 4, 48, 7, 21, 47], but only a small number have been demonstrated to be applicable even to the large-scale DNN models commonly employed in real-world computer vision tasks.”