Sampling from multimodal distributions using tempered transitions

By Radford M. Neal. 1996

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Better Mixing via Deep Representations (2012, methodology): “To improve mixing, a powerful idea that has been explored recently for deep learning algorithms (Desjardins et al. , 2010; Cho et al. , 2010; Salakhutdinov, 2010b,a) is tempering (Neal, 1994).”
Generalizing Hamiltonian Monte Carlo with Neural Networks (2017, methodology): “It is further capable of emulating several common extensions of HMC such as withintrajectory tempering (Neal, 1996) and diagonal mass matrices (Bennett, 1975).”
Annealed importance sampling (1998, methodology): “Annealed importance sampling is related to tempered transitions (Neal 1996), which are another way of modifying the annealing procedure so as to produce correct results.”
Handbook of Markov Chain Monte Carlo (2011, positive): “Statistical applications of HMC began with my use of it for neural network models (Neal, 1996a).”
Accurate and conservative estimates of MRF log-likelihood using reverse annealing (2014, background): “The values of β can be spaced evenly between 0 and 1, although other schedules have been explored (Neal, 1996; Behrens et al., 2012; Calderhead and Girolami, 2009).”
Improving the acceptance rate of reversible jump MCMC proposals (2004, background): “(For more discussion of tempering see Neal, 1996.)”
Shadow hybrid Monte Carlo: an efficient propagator in phase space of macromolecules (2004, methodology): “HMC is combined with tempering by Neal [33].”
Parallel Tempering for Training of Restricted Boltzmann Machines (2010, methodology): “In work which was done concurrently but independently of ours, (Salakhutdinov, 2010) uses the method of tempered transitions (Neal, 1994) to promote mixing and fine-tuning the parameters of Markov random fields.”
Bayesian State-Space Modelling on High-Performance Hardware Using LibBi (2013, background): “Adaptation (Gilks, Roberts, and Sahu 1998; Haario, Saksman, and Tamminen 2001; Andrieu and Thoms 2008) and tempering (Frantz, Freeman, and Doll 1990; Geyer 1991; Marinari and Parisi 1992; Geyer and Thompson 1995; Neal 1996) are both established means of reducing Bθ for single chains.”
JAGS Version 3.3.0 user manual (2012, background): “It uses tempered transitions to jump between distant modes of the multi-modal posterior distribution generated by such models [8, 2].”
Foundations and Advances in Deep Learning (2014, methodology): “Salakhutdinov (2009) proposed to use tempered transitions (Neal, 1994), while a similar, but not identical sampling method based on tempered chains called parallel tempering was proposed in Publication III and independently by Desjardins et al.”
The Controlled Thermodynamic Integral for Bayesian Model Comparison (2014, background): “Neal (1996) showed that a geometric temperature ladder is optimal for annealing on the scale parameter of a Gaussian and Behrens et al. (2012) extended this result to target distributions of the same form as g(θ), which includes Gaussians.”
MCMC Using Hamiltonian Dynamics (2011, background): “C. However, temperinglike behaviour can also be incorporated directly into trajectory used to propose a new state in the HMC procedure. In the simplest version of such a “tempered trajectory” scheme (Neal, 1996b, Section 6), each leapfrog step in the ﬁrst half of the trajectory is combined with mult”
Subspace Inference for Bayesian Deep Learning (2019, methodology): “Tempered posteriors are often used in Bayesian inference algorithms to enhance multi-modal explorations (e.g., Geyer and Thompson, 1995; Neal, 1996b).”
Estimating Ratios of Normalizing Constants Using Linked Importance Sampling (2005, methodology): “AIS is related to an earlier method for moving between isolated modes that I call ‘tempered transitions’ (Neal 1996).”