Adaptive importance sampling in general mixture classes

By Olivier Capp'e, Randal Douc, Arnaud Guillin, Jean-Michel Marin, Christian P. Robert. 2007

319 citations recorded by the source corpus.

In this paper, we propose an adaptive algorithm that iteratively updates both the weights and component parameters of a mixture importance sampling density so as to optimise the performance of importance sampling, as measured by an entropy criterion. The method, called M-PMC, is shown to be applicable to a wide class of importance sampling densities, which includes in particular mixtures of multivariate Student t distributions. The performance of the proposed scheme is studied on both artificial and real examples, highlighting in particular the benefit of a novel Rao-Blackwellisation device which can be easily incorporated in the updating scheme.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Kernel Sequential Monte Carlo (2015, methodology): “In this paper, we focus on static SMC methods, which recently have generated increasing interest as an alternative to MCMC for Bayesian inference on a single target distribution [3, 4, 6, 11].”
Adaptive Multiple Importance Sampling (2009, methodology): “For instance, if the potential for multimodality of the target Π is high enough, a mixture of Student’s t distributions as in Cappé et al. (2008) would be more appropriate. When dealing with a Bayesian hierarchical model, creating classes (or blocks) of components of the parameter in agreement with ”
An Adaptive Sequential Monte Carlo Sampler (2010, background): “Further related work includes that of Douc et al. (2005) and Capp´e et al. (2008) on respectively population Monte Carlo and adaptive importance sampling.”
Adaptive approximate Bayesian computation (2008, background): “We have thus established that the adaptive scheme of Cappé et al. (2008) is also appropriate in this setting, towards a better fit of the proposal kernel Kt to the target π{θ | %(x, y) < }.”
Sequential Monte Carlo on large binary sampling spaces (2011, methodology): “Besides Sequential Monte Carlo, important classes of adaptive Monte Carlo are Adaptive Importance Sampling (e.g. Capp´e et al., 2008) and Adaptive Markov chain Monte Carlo (e.g. Andrieu and Thoms, 2008).”
Galaxy clustering in the CFHTLS-Wide: the changing relationship between galaxies and haloes since z ~ 1.2 (2011, methodology): “Contrary to the widely-used Monte Carlo Markov Chain method (MCMC), PMC is an adaptive importance-sampling technique (Cappé et al. 2004, 2008), which has two principal advantages: first, the parallel sampling algorithm combined with our fast HOD code means that each run can be quickly computed.”
Variational Inference with Tail-adaptive f-Divergence (2018, background): “…(De Boer et al., 2005); χ 2 -divergence is exploited for VI (e.g., Dieng et al., 2017), but is more extensively studied in the context of adaptive importance sampling (IS) (e.g., Cappé et al., 2008; Ryu & Boyd, 2014; Cotter et al., 2015), since it coincides with the variance of the IS…”
On the Asymptotic Efficiency of Approximate Bayesian Computation Estimators (2015, methodology): “q k+1(), a simple criteria is the dierence k k k+1k+j k k+1j1=2. Besides constructing q k() as a unimodal density, other methods of constructing the importance proposal can be applied including [37,10,44,29]. Since algorithm 2 has the same simulation size as the rejection ABC and the additional cal”
ZhuSuan: A Library for Bayesian Deep Learning (2017, background): “Therefore people have been exploring the use of adaptive proposals (Capp´e et al., 2008; Cornuet et al., 2012).”
A tutorial on adaptive MCMC (2008, background): “7) to consider mixtures, finite or infinite, of distributions belonging to the exponential family (see also Cappé et al. 2007 for a similar idea in the context of importance sampling/population Monte Carlo).”
Inference Networks for Sequential Monte Carlo in Graphical Models (2016, methodology): “Recently proposed methods for Bayesian inference based on sequential Monte Carlo (Doucet et al., 2001) have shown themselves to provide state-of-the art results in applications far broader than the traditional use of sequential Monte Carlo (SMC) for filtering in state space models (Gordon et al., 19”
Generalized Multiple Importance Sampling (2015, methodology): “In other techniques, such as mixture PMC (M-PMC) (Douc et al., 2007a, 2007b, Cappé et al., 2008), a similar strategy is employed, but using sampling S1 in the mixture φt(x), that is, with the MIS scheme R3.”
CFHTLenS: combined probe cosmological model comparison using 2D weak gravitational lensing (2012, methodology): “PMC is an adaptive importance-sampling technique (Cappé et al. 2004, 2008) in which samples pn, n = 1 . . .”
Gradient Importance Sampling (2015, methodology): “After collecting new samples, D-Kernel approximations [16] or Mixture distributions (such as Mixtures of Gaus-sians or Student-T mixtures) [17] are updated to ﬁt the posterior better.”
Evidence of the accelerated expansion of the Universe from weak lensing tomography with COSMOS (2009, methodology): “This algorithm is an adaptive importance-sampling technique (Cappé et al. 2007): instead of creating a sample under the posterior as done in traditional Monte-Carlo Markov chain (MCMC) techniques (e.g. Christensen et al. 2001), points are sampled from a simple distribution, the so-called proposal,…”
Rethinking the Effective Sample Size (2018, background): “…ðwÞ. β ¼ 0 β ¼ 1=2 β ¼ 1 β ¼ 2 β ¼ ∞ jjwjj0 ∑ N n¼1 ffiffiffiffiffi wn p 2 exp ∑ N n¼w wn logwn 1 ∑Nn¼1w 2 n 1 max w1 ; …; wN ½ L0 norm of w (number of non-zero elements) (perplexity) (Cappé et al., 2008) dESS (Kong, 1992) (Martino et al., 2017) [Correction added on 25 April 2022, after first…”