Non-linear regression models for Approximate Bayesian Computation

By Michael G. B. Blum, Olivier Francois. Statistics and computing, 2008

494 citations recorded by the source corpus.

Approximate Bayesian inference on the basis of summary statistics is well-suited to complex problems for which the likelihood is either mathematically or computationally intractable. However the methods that use rejection suffer from the curse of dimensionality when the number of summary statistics is increased. Here we propose a machine-learning approach to the estimation of the posterior density by introducing two innovations. The new method fits a nonlinear conditional heteroscedastic regression of the parameter on the summary statistics, and then adaptively improves estimation using importance sampling. The new algorithm is compared to the state-of-the-art approximate Bayesian methods, and achieves considerable reduction of the computational burden in two examples of inference in statistical genetics and in a queueing model.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/d7633503b0f45f491ccbca8a7826d918dae974c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, methodology): “A second perspective on the ABC output is that it is based on a non-parametric approximation of the sampling distribution (Blum 2010; Blum and Fran¸cois 2010), connected with both indirect inference (Drovandi et al. 2011) and k - nearest neighbour estimation (Biau et al. 2014).”
A Novel Approach for Choosing Summary Statistics in Approximate Bayesian Computation (2012, methodology): “302 2002 , Blum and François 2010 , Leuenberger and Wegmann 2010 ) and scaling of parameters .”
A comparative review of dimension reduction methods in approximate Bayesian computation (2012, methodology): “A number of methods, primarily based on dimension reduction ideas, have been proposed to achieve this (Joyce and Marjoram 2008; Wegmann et al. 2009; Nunes and Balding 2010; Blum and François 2010; Blum 2010b; Fearnhead and Prangle 2012).”
Supervised Machine Learning for Population Genetics: A New Paradigm (2018, methodology): “Work has been done to retain more of the simulations in ABC, for instance by weighing their influence on parameter estimation according to their similarity to the observed data [62].”
ABC-EP: Expectation Propagation for Likelihoodfree Bayesian Computation (2011, methodology): “Del Moral et al. (2008) adapt sequential Monte Carlo techniques (Del Moral et al.”
Bayesian computation: a summary of the current state, and samples backwards and forwards (2015, methodology): “A second perspective on the ABC output is that it is based on a nonparametric approximation of the sampling distribution (Blum 2010; Blum and François 2010), connected with both indirect inference (Drovandi et al.”
Reconstructing routes of invasion using genetic data: why, how and so what? (2010, methodology): “Recent improvements of ABC get round this problem by using dimension reduction techniques, including a nonlinear feed-forward neural network (Blum & François 2010) and partial least squares regression (Wegmann et al. 2009).”
Approximate Bayesian computation via regression density estimation (2012, methodology): “Leuenberger and Wegmann (2010) and other regression adjustment postprocessing methods (Beaumont et al. 2002; Blum and Francois 2010; Nott et al. 2012) for further information.”
Flexible statistical inference for mechanistic models of neural dynamics (2017, methodology): “Finally, methods originating from regression correction [33, 18, 19] simulate multiple data xn from different θn sampled from a proposal distribution p̃(θ), and construct a conditional density estimate q(θ|x) by performing a regression from simulated data xn to θn.”
The Rate of Convergence for Approximate Bayesian Computation (2013, methodology): “…method include placing ABC in an MCMC context (Marjoram et al., 2003), sequential ABC (Sisson et al., 2007), enhancing ABC with nonlinear regression models (Blum and Fran¸cois, 2010), agent-based modelling (Sottoriva and Tavar´e, 2010), and a Gibbs sampling ABC approach (Wilkinson et al., 2011).”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, methodology): “…& Sederberg, 2014; Papamakar-ios et al., 2018; Lueckmann et al., 2018) or posterior (Le et al., 2017; Papamakarios & Murray, 2016; Lueckmann et al., 2017; Chan et al., 2018) (which can be traced back to regression-adjustment, Beaumont et al., 2002; Blum & Franc¸ois, 2010) are reviewed above (2.3).”
On the Asymptotic Efficiency of Approximate Bayesian Computation Estimators (2015, methodology): “oice of the proposal distribution is key to the performance of the importance sampling. [17] used a pilot stage to nd the high posterior density region for constructing the proposal distribution, and [7] used iterative procedure to learn good proposal distributions. However, as it is closer to the p”
Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network (2015, methodology): “The FFNN used here is diﬀerent than that used by Blum and Fran¸cois (2010).”
Likelihood-free inference with emulator networks (2018, background): “Approximate Bayesian Computation (ABC) aims to close this gap by providing Bayesian inference algorithms that can be used with simulator models without requiring explicit likelihoods [3, 4, 5, 6, 7].”
Optimization Monte Carlo: Efficient and Embarrassingly Parallel Likelihood-Free Inference (2015, unknown): “6 Bayesian Inference of the M/G/1 Queue Model Bayesian inference of the M/G/1 queuing model is challenging, requiring ABC algorithms [4, 8] or sophisticated MCMC-based procedures [18].”