On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods

By Anthony Lee, Christopher Yau, M. Giles, Arnaud Doucet, Christopher C. Holmes. Journal of Computational And Graphical Statistics, 2009

326 citations recorded by the source corpus.

We present a case study on the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. Graphics cards, containing multiple Graphics Processing Units (GPUs), are self-contained parallel computational devices that can be housed in conventional desktop and laptop computers and can be thought of as prototypes of the next generation of many-core processors. For certain classes of population-based Monte Carlo algorithms they offer massively parallel simulation, with the added advantage over conventional distributed multicore processors that they are cheap, easily accessible, easy to maintain, easy to code, dedicated local devices with low power consumption. On a canonical set of stochastic simulation examples including population-based Markov chain Monte Carlo methods and Sequential Monte Carlo methods, we find speedups from 35- to 500-fold over conventional single-threaded computer code. Our findings suggest that GPUs have the potential to facilitate the growth of statistical modeling into complex data-rich domains through the availability of cheap and accessible many-core computation. We believe the speedup we observe should motivate wider use of parallelizable simulation methods and greater methodological attention to their design. This article has supplementary material online.

View Paper: https://www.semanticscholar.org/paper/f74e04c23db19d746096eec7821f10112690f392

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach (2013, methodology): “These methods admit natural and scalable parallelisation and we demonstrate the potential of these algorithms with real implementations suitable for use on consumer-grade parallel computing hardware including GPUs, reinforcing the message of Lee et al. (2010).”
Exploiting Multi-Core Architectures for Reduced-Variance Estimation with Intractable Likelihoods (2014, background): “…can be performed in parallel, oﬀering the opportunity to exploit modern multi-core processing architectures (Suchard et al., 2010; Lee et al., 2010) in a straightforward manner that directly complements (and is compatible with) related research eﬀorts for parallelisation of MCMC…”
Bayesian State-Space Modelling on High-Performance Hardware Using LibBi (2013, background): “SMC is particularly promising in this regard (Lee et al. 2010).”
On the stability of sequential Monte Carlo methods in high dimensions (2011, methodology): “, [36]) SMC methods are applicable, in the static context, in high-dimensions.”
Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, background): “While research on MCMC theory and methodology is still active and continually branching (Papaspiliopoulos et al. 2007; An-drieu and Roberts 2009; (cid:32)Latuszy´nski et al. 2011; Douc and Robert 2011), progressively incorporating the capacities of parallel processors and GPUs (Lee et al. 2009; Jaco”
Biips: Software for Bayesian Inference with Interacting Particle Systems (2014, methodology): “Nonetheless, there is room for improvement and optimization of Biips ; in particular, particle algorithms are particularly suited to parallelization (Lee et al. 2010; Vergé et al. 2013; Murray 2013), and we plan in future releases of the software to provide a parallel implementation of Biips .”
Filtering via approximate Bayesian computation (2010, unknown): “When one considers the developments in GPU technology (Lee et al. 2009) and the simplicity of the model, this type of strategy is feasible from an algorithmic trading perspective (based upon personal communication with industry professionals.)”
On the role of interaction in sequential Monte Carlo algorithms (2013, unknown): “Keywords: convergence; hidden Markov model; particle filters”
Uniform ergodicity of the iterated conditional SMC and geometric ergodicity of particle Gibbs samplers (2013, background): “Moreover, this improvement can often be obtained with little extra computational effort, since on a parallel architecture one can sample from M and evaluate G in parallel, a characteristic of SMC algorithms more generally [17].”
Joint estimation of multiple networks from time course data (2013, methodology): “Advances in MCMC and parallel computing may in the future ameliorate these issues (Lee et al., 2010), but at present it remains the case that fast, interactive joint estimation is currently challenging or prohibitively demanding using MCMC. We therefore propose an exact approach, using an in-degree ”
Bayesian computation: a summary of the current state, and samples backwards and forwards (2015, background): “While research on MCMC theory and methodology is still active and continually branching (Papaspiliopoulos et al. 2007; Andrieu and Roberts 2009; Łatuszy´nski et al. 2011; Douc and Robert 2011), progressively incorporating the capacities of parallel processors and GPUs (Lee et al. 2009; Jacob et al. ”