Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach

By Rémi Bardenet, Arnaud Doucet, Christopher C. Holmes. 2014

178 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Kernel Test of Goodness of Fit (2016, methodology): “In our experiments, a particular focus is on applying our goodness-of-fit test to certify the output of approximate Markov Chain Monte Carlo (MCMC) samplers (Korattikara et al., 2014; Welling & Teh, 2011; Bardenet et al., 2014).”
Stochastic Gradient Hamiltonian Monte Carlo (2014, positive): “The efficiency of this MH step could potentially be improved using the recent results of (Korattikara et al., 2014; Bardenet et al., 2014).”
Automated Scalable Bayesian Inference via Hilbert Coresets (2017, methodology): “, 2015), subsampling methods for MCMC (Welling and Teh, 2011; Ahn et al., 2012; Bardenet et al., 2014; Korattikara et al., 2014; Maclaurin and Adams, 2014; Bardenet et al., 2015), and distributed “consensus” methods for MCMC (Scott et al.”
On the Use of Cauchy Prior Distributions for Bayesian Logistic Regression (2015, methodology): “For example, posterior means under Cauchy priors estimated via Monte Carlo and other approximations have been reported in Bardenet et al. (2014); Chopin and Ridgway (2015). It is well-known that the mean does not exist for the Cauchy distribution, so clearly the prior means of the regression coeffic”
Stochastic Expectation Propagation (2015, background): “Examples include sampling approximations [1, 2], distributional approximations including stochastic variational inference (SVI) [3] and assumed density filtering (ADF) [4], and approaches that mix distributional and sampling approximations [5, 6].”
A Complete Recipe for Stochastic Gradient MCMC (2015, methodology): “A similar bias-speed tradeoff was used in [11, 3] to construct MH samplers, in addition to being used in SGLD and SGHMC. 3 Applying the Theory to Construct Samplers”
Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning (2019, background): “To correct this error, it is possible to do stochastic Metropolis-Hastings (MH) (Korattikara et al., 2014; Bardenet et al., 2014; Chen et al., 2016b).”
Distributed Bayesian Learning with Stochastic Natural Gradient Expectation Propagation and the Posterior Server (2015, methodology): “…and Shang, 2015, Ma et al., 2015, Li et al., 2016a) which uses minibatch stochastic gradients within MCMC, austerity MCMC (Korattikara et al., 2014, Bardenet et al., 2014) which uses data subsampling to reduce computational cost of Metropolis-Hastings acceptance steps, and embarrassingly parallel…”
Modeling, Inference and Optimization With Composable Differentiable Procedures (2016, background): “[7] have shown that such approximate MH moves can lead to stationary distributions which are approximate but that have bounded error, albeit under strong conditions of rapid mixing.”
Stochastic Gradient Markov Chain Monte Carlo (2019, methodology): “Alternatively, rather than using multiple computer cores, a single MCMC algorithm can be used, where only a subsample of the data is evaluated at each iteration (Bardenet et al. 2017). For example, in the Metropolis-Hastings algorithm, the accept-reject step can be approximated with a subset of the ”
Exploiting Multi-Core Architectures for Reduced-Variance Estimation with Intractable Likelihoods (2014, background): “…et al., 2010) in a straightforward manner that directly complements (and is compatible with) related research eﬀorts for parallelisation of MCMC methodology (Alquier et al., 2014; Angelino et al., 2014; Bardenet et al., 2014; Calderhead, 2014; Korattikara et al., 2014; Maclaurin and Adams, 2014).”
Bayesian Sampling Using Stochastic Gradient Thermostats (2014, background): “There is some recent work [2, 3, 14] which attempts to estimate the Metropolis step using partial data.”
Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, background): “…complex models with intractable like-lihoods, but also by the need of MCMC algorithms for Big Data in contexts where the likelihood function can not be evaluated for the whole dataset (Ko-rattikara et al. 2013; Bardenet et al. 2014; Teh et al. 2014; Maclaurin and Adams 2014; Minsker et al. 2014).”
Firefly Monte Carlo: Exact MCMC with Subsets of Data (2014, background): “(2014) and Bardenet et al. (2014) have shown that such approximate MH moves can lead to stationary distributions which are approximate but that have bounded error, albeit under strong conditions of rapid mixing.”
Sparse Variational Inference: Bayesian Coresets from Scratch (2019, methodology): “Other scalable Bayesian inference algorithms have largely been developed by modifying standard inference algorithms to handle distributed or streaming data processing [10, 11, 18–29], which tend to have no guarantees on inferential quality and require extensive model-specific expert tuning.”