Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network

By Bai Jiang, Tung-Yu Wu, Charles Yang Zheng, Wing Hung Wong. 2015

158 citations recorded by the source corpus.

Approximate Bayesian Computation (ABC) methods are used to approximate posterior distributions in models with unknown or computationally intractable likelihoods. Both the accuracy and computational efficiency of ABC depend on the choice of summary statistic, but outside of special cases where the optimal summary statistics are known, it is unclear which guiding principles can be used to construct effective summary statistics. In this paper we explore the possibility of automating the process of constructing summary statistics by training deep neural networks to predict the parameters from artificially generated data: the resulting summary statistics are approximately posterior means of the parameters. With minimal model-specific tuning, our method constructs summary statistics for the Ising model and the moving-average model, which match or exceed theoretically-motivated summary statistics in terms of the accuracies of the resulting posteriors.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

Citations in the corpus

Towards a Neural Statistician (2016, background): “More recently Jiang et al. [12] used deep neural networks to predict the parameters generating the data.”
BayesFlow: Learning Complex Stochastic Models With Invertible Neural Networks (2020, methodology): “We compare the performance of BayesFlow to the following recent methods capable of amortized likelihood-free inference: conditional variational autoencoder (cVAE) [38], cVAE with autoregressive flow (cVAE-IAF) [27], DeepInference with heteroscedastic loss [44], approximate Bayesian computation with ”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, background): “Other works investigated learning summary statistics (Fearnhead & Prangle, 2012; Dinev & Gutmann, 2018; Wong et al., 2018).”
Flexible statistical inference for mechanistic models of neural dynamics (2017, background): “Learning features: Finally, we point out that using a neural network for posterior estimation yields a straightforward way of learning relevant features from data [38, 39, 40].”
Likelihood-Free Inference by Ratio Estimation (2016, unknown): “While there are several works on summary statistics selection in the framework of approximate Bayesian computation (Aeschbacher et al., 2012; Fearnhead and Prangle, 2012; Blum et al., 2013; Gutmann et al., 2014; Marin et al., 2016; Gutmann et al., 2018; Jiang et al., 2018), there is comparably littl”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, methodology): “…virtually all algorithms rely on summary statistics in order to reduce the dimensionality of the data (Fearnhead & Prangle, 2012; Blum et al., 2013; Jiang et al., 2017)—designing summary statistics can require domain-speciﬁc knowledge and/or separate optimization procedures, and might bias…”
Supervised Machine Learning for Population Genetics: A New Paradigm (2018, methodology): “Interestingly, several groups have experimented with augmenting ABC by using ML for selecting the optimal combination of summary statistics [60] or even generating them [61].”
INFERNO: Inference-Aware Neural Optimisation (2018, methodology): “[23] successfully employ a summary statistic by directly regressing the parameters of interest and therefore approximating the posterior mean given the data, which then can be used directly as a summary statistic.”