Optimal scaling of discrete approximations to Langevin diffusions

By Gareth O. Roberts, Jeffrey S. Rosenthal. 1998

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, methodology): “For image space discriminative models, plug-and-play generative networks [66] control the attributes of generated images via Langevin dynamics [78]; these ideas are also explored in diffusion models [86].”
Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, methodology): “We sample from the resulting combined model by following gradients in the latent representation space in a manner inspired by the approximate Metropolis-adjusted Langevin (MALA) (Roberts et al., 1996; Roberts & Rosenthal, 1998) sampler deployed in Nguyen et al. (2017).”
Handbook of Markov Chain Monte Carlo (2011, neutral): “…a nearly independent point using LMC is proportional to 1/(a(µ)ε2), and since µ is proportional to ε6, we can write the cost as CLMC ∝ 1 / (a(µ)µ1/3) (5.18) Numerical calculation shows that this is minimized when a(µ) is 0.57 a result obtained more formally by Roberts and Rosenthal (1998).”
A Tutorial on Thompson Sampling (2017, background): “A number of papers establish rigorous guarantees for the rate at which this Markov chain converges to its stationary distribution [20, 21, 22, 23].”
Optimal Proposal Distributions and Adaptive MCMC (2008, methodology): “Roberts and Rosenthal (1998) proved that in this case, under the same i.i.d. target assumption (2), a similar optimal scaling result holds.”
Norges Teknisk-naturvitenskapelige Universitet Control Variates for the Metropolis-hastings Algorithm Control Variates for the Metropolis-hastings Algorithm (background): “The second is the Langevin proposal (Roberts and Rosenthal, 1998) q L (y|x) = N 10 (y|x + σ 2 2 ∇ log{π(x)}, σ 2 I).”
An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants (2006, background): “Recommendations on how to tune Metropolis-Hastings algorithms to obtain optimal acceptance probabilities may exist in the case of (4) (see e.g. Gelman et al. (1996), Roberts et al. (1997), Roberts and Rosenthal (1998) and Breyer and Roberts (2000)).”
Langevin diffusions and the Metropolis-adjusted Langevin algorithm (2013, methodology): “Scaling properties of h with d and asymptotic optimal acceptance rates for the method are discussed in Roberts and Rosenthal (1998).”
Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, methodology): “For MALA algorithms an optimal acceptance rate of 0.574 has been established in Roberts and Rosenthal (1998) and con-ﬁrmed in inﬁnite-dimensional settings in Pillai et al. (2012) ) .”
Optimal scaling of Metropolis algorithms: Heading toward general target distributions (2008, methodology): “…has also been applied to derive optimal scaling results for other types of MCMC algorithms, such as the Metropolis-adjusted Langevin algorithm (MALA); see Roberts and Rosenthal (1998, 2001), Breyer, Piccioni, and Scarlatti (2002), Christensen, Roberts, and Rosenthal (2003), Neal and Roberts (2006).”
Optimal Scaling and Diffusion Limits for the Langevin Algorithm in High Dimensions (2011, methodology): “1), the optimality of the exponent γ = (1)3 was first obtained in [23].”
Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions (2003, methodology): “This corroborates the empirical observations of Troughton and Godsill (2001), who suggested that using the conditional distribution to propose the value of new parameters is particularly efficient and they went on to show that the proposal and acceptance ratio can be simplified in this case.”
Gen: a general-purpose probabilistic programming system with programmable inference (2019, methodology): “Probabilistic Inference Gen provides language constructs, DSLs, and inference library routines for state-of-the-art techniques from probabilistic modeling and inference, including generative modeling [32], deep neural networks [1], Monte Carlo inference [2, 16, 38], and numerical optimization [17].”
Optimal tuning of the hybrid Monte Carlo algorithm (2010, unknown): “• There is evidence that the limiting properties of MALA for high-dimensional target densities do not appear to depend critically on the tail behaviour of the target (see [23]).”
Weak convergence of Metropolis algorithms for non-i.i.d. target distributions (2007, background): “Optimal scaling issues have been explored by other authors, namely [5, 6, 10, 12, 13].”