Monte Carlo methods

By Rémi Bardenet. 2013

1,800 citations recorded by the source corpus.

Bayesian inference often requires integrating some function with respect to a posterior distribution. Monte Carlo methods are sampling algorithms that allow to compute these integrals numerically when they are not analytically tractable. We review here the basic principles and the most common Monte Carlo algorithms, among which rejection sampling, importance sampling and Monte Carlo Markov chain (MCMC) methods. We give intuition on the theoretical justification of the algorithms as well as practical advice, trying to relate both. We discuss the application of Monte Carlo in experimental physics, and point to landmarks in the literature for the curious reader.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Consistent Kernel Mean Estimation for Functions of Random Variables (2016, methodology): “For strings, probabilistic ﬁnite automata are often used. All those approaches have their merits, but they only work with a speciﬁc input type. There is an alternative, based on Monte Carlo sampling (Kalos and Whitlock, 2008), which is to represent Xby a (possibly weighted) sample f(x i;w i)gn i=1 (”
RMM: Reinforced Memory Management for Class-Incremental Learning (2023, methodology): “According to the policy gradient theorem [51], we can compute the gradients for J ( η, φ ) as follows, Following the REINFORCE algorithm [51], we replace the expectations E T [ · ] and E π η ,π φ [ · ] with sample averages using the Monte Carlo method [16].”
Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning (2019, background): “IPS has a rich history in statistics [44, 20, 24], with successful crossover to RL [45].”
Estimating the Marginal Likelihood Using the Arithmetic Mean Identity (2017, methodology): “Meng and Wong (1996), Raftery (1996), Lewis and Raftery (1997), Han and Carlin (2001), Raftery et al.”
Crowdsourcing in Computer Vision (2016, unknown): “textbooks by Kalos and Whitlock [59], and Spanier and Gelbard [114].”
A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks (2022, methodology): “This study also considers other low-discrepancy sequences, including the Halton sequence [32] and the Hammersley sequence [33].”
Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning (2017, methodology): “• For variance reduction, a common strategy is to use an additive control variate baseline [16, 12, 11].”
Bayesian updating via bootstrap filtering combined with data-driven polynomial chaos expansions: methodology and application to history matching for carbon dioxide storage in geological formations (2013, background): “, [47], [69]), because even single deterministic simulations may require parallel high-performance computing.”
Parallel random numbers: As easy as 1, 2, 3 (2011, unknown): “critical to a wide variety of applications [19].”
Efficient training of physics‐informed neural networks via importance sampling (2021, background): “Among the popular lowdiscrepancy sequences are the generalized Halton sequences [45, 46], the Sobol sequences [47, 48], and the Hammersley sets [49, 50].”
Model Comparison and the Principle of Parsimony (2015, unknown): “This problem can be overcome by a numerical technique known as importance sampling (Hammersley & Handscomb, 1964).”
Monte Carlo convolution for learning on non-uniformly sampled point clouds (2018, methodology): “Since we only have a set of samples of our function f , we propose to compute this integral by using MC integration [Kalos and Whitlock 1986], which uses a set of random samples to compute the value of an integral.”