Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference

By Nicolas Chopin. 2004

485 citations recorded by the source corpus.

The term sequential Monte Carlo methods or, equivalently, particle filters, refers to a general class of iterative algorithms that performs Monte Carlo approximations of a given sequence of distributions of interest (π t ). We establish in this paper a central limit theorem for the Monte Carlo estimates produced by these computational methods. This result holds under minimal assumptions on the distributions π t , and applies in a general framework which encompasses most of the sequential Monte Carlo methods that have been considered in the literature, including the resample-move algorithm of Gilks and Berzuini [J. R. Stat. Soc. Ser. B Stat. Methodol. 63 (2001) 127-146] and the residual resampling scheme. The corresponding asymptotic variances provide a convenient measurement of the precision of a given particle filter. We study, in particular, in some typical examples of Bayesian applications, whether and at which rate these asymptotic variances diverge in time, in order to assess the long term reliability of the considered algorithm.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Filtering with State-Observation Examples via Kernel Monte Carlo Filter (2013, methodology): “generating samples Yi ∼ p(·|Xi) (i = 1,...,n) and making an estimator by Pˆ Y = 1 n Pn i=1δY i(·). Convergence results for such estimators have been obtained in the particle methods literature (e.g., Chopin, 2004). In the following, we will show that the same sampling procedure for the estimator mˆP”
Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem (2022, methodology): “Although the proof provided in [6] is restricted to the simpler, but higher variance, case where the resampling step uses multinomial resampling, the authors note that [5] proves it holds in the case of residual resampling (which we use in our experiments) as well.”
Accurate Methods for Approximate Bayesian Computation Filtering (2015, background): “(1.1) Under a wide range of conditions, the sample mean P t ) converges to E Φ( x t ) | y 1: t } as N → ∞ for any bounded measurable function Φ (Chopin, 2004; Crisan & Doucet, 2002).”
Towards Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach (2013, methodology): “This annealing scheme has been used in Del Moral et al. (2006b); Jasra et al. (2007a) and many other works.”
Nested Sequential Monte Carlo Methods (2015, methodology): “0 with E k k [kW0fk2+] <1and such that the function (x 1: k 1;u 0: ) 7!E QM k [W0 kf] is in k 1. Furthermore, assume that the identity function f 1 belongs to k for each k. Then, it follows by Chopin (2004, Theorem 1 and Lemma A.1) that N1=2 XN i=1 1 N f(Xi 1:k;U i 0:k) E k [f]) ! ! ND (0;VeM k (”
Monte Carlo Confidence Sets for Identified Sets (2016, unknown): “21See Chopin (2002, 2004) and Del Moral, Doucet, and Jasra (2006) for the generic SMC algorithm for estimating static model parameters.”
Sequential quasi Monte Carlo (2015, background): “Actually, lower variance solutions that amount to qMC, like systematic and residual sampling (Gundersen and Jensen, 1987; Fearnhead, 1998; Chopin, 2004), have been proposed in the particle literature.”
Negative association, ordering and convergence of resampling methods (2017, methodology): “For sake of completeness, Theorem 7 also presents results for the multinomial resampling (ρmulti) and residual reampling (ρres/multi) schemes for which a central limit theorem also exists (see Chopin, 2004; Künsch, 2005; Douc et al., 2005).”
Comparison of resampling schemes for particle filtering (2005, methodology): “The variance formula given in Corollary 5 was first derived in [3] which however lacked a rigorous proof of Lemma 4 and the necessity of the support condition – see [6] for a counter-example showing that this condition is indeed necessary and non-trivially satisfied.”
On the stability of sequential Monte Carlo methods in high dimensions (2011, background): “, [19, 21, 25, 26, 30, 35]) they do not consider this latter scenario.”
Dynamic Prediction Pools: An Investigation of Financial Frictions and Forecasting Performance (2016, unknown): “The SW model is based on earlier work by Christiano et al. (2005) and Smets and Wouters (2003).”
An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo (2007, background): “The degree to which this assertion is correct is discussed in [41], [42], [43], [44], [32], [7] but we only need to know at this stage that the general intuition is justifiable.”
On sequential Monte Carlo, partial rejection control and approximate Bayesian computation (2008, unknown): “Central Limit Theorems (CLTs) for SMC and particle filtering algorithms have been derived in various articles (Del Moral 2004; Kunsch 2005; Chopin 2004; Del Moral et al. 2006; Johansen and Doucet 2008).”
Particle filters for partially observed diffusions (2007, methodology): “Appendix F: Central Limit Theorem For notational simplicity, we consider a special case of our particle filter, chosen to resemble those considered in Chopin (2004). We choose our proposal density for time ti+1 to have βj = w (j) i ; and we assume iid sampling of X (j) ti in step PF1. The particle f”
Importance Sampling Squared for Bayesian Inference in Latent Variable Models (2013, methodology): “cles {(θi,zi)} Np i=1 produced by Algorithm 2’, we have that bπAS(ϕ) = 1 Np XNp i=1 ϕ(θi) = 1 Np XNp i=1 ϕe(θi,zi) −→a.s. π N(ϕe) = π(ϕ). (19) In addition, if ϕ satisﬁes the condition in Theorem 1 of Chopin (2004), we have that p Np bπAS(ϕ)−π(ϕ) → Nd (0,σ2 AS(ϕ)) as Np → ∞, (20) with the asymptoti”