A comparative review of dimension reduction methods in approximate Bayesian computation

By Michael G. B. Blum, Matthew A. Nunes, Dennis Prangle, Scott Anthony Sisson. 2012

376 citations recorded by the source corpus.

Approximate Bayesian computation (ABC) methods make use of comparisons between simulated and observed summary statistics to overcome the problem of computationally intractable likelihood functions. As the practical implementation of ABC requires computations based on vectors of summary statistics, rather than full data sets, a central question is how to derive low-dimensional summary statistics from the observed data with minimal loss of information. In this article we provide a comprehensive review and comparison of the performance of the principal methods of dimension reduction proposed in the ABC literature. The methods are split into three nonmutually exclusive classes consisting of best subset selection methods, projection techniques and regularization. In addition, we introduce two new methods of dimension reduction. The first is a best subset selection method based on Akaike and Bayesian information criteria, and the second uses ridge regression as a regularization procedure. We illustrate the performance of these dimension reduction techniques through the analysis of three challenging models and data sets.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b537e79c16b5d9a5a5546a1b8aeb5e8887d50495

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Efficient Acquisition Rules for Model-Based Approximate Bayesian Computation (2017, background): “The discrepancy and the summaries affect the approximations and their choice is an active research topic (Blum et al., 2013; Fearnhead and Prangle, 2012; Gutmann et al., 2017).”
Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, background): “In the ABC literature, they are called summary statistics , a term that does not impose any constraint on their form and hence leaves open the question of performance, as discussed in Marin et al. (2011); Blum et al. (2013).”
Dynamic Likelihood-free Inference via Ratio Estimation (DIRE) (2018, methodology): “eduction methods and methods from arXiv:1810.09899v1 [stat.ML] 23 Oct 2018 Traiko Dinev, Michael U. Gutmann regression and classication (see e.g. Aeschbacher et al. 2012; Fearnhead and Prangle 2012; Blum et al. 2013; Gutmann et al. 2018; Jiang et al. 2018). But for the synthetic likelihood approach ”
Reliable ABC model choice via random forests (2014, background): “Second, calibration has always been a critical step in ABC implementation (Blum et al., 2013; Marin et al., 2012).”
Supervised Machine Learning for Population Genetics: A New Paradigm (2018, methodology): “Most importantly, when using large feature vectors, ABC is susceptible to the curse of dimensionality [59] – much effort has therefore gone into dimensionality reduction and feature selection for ABC (reviewed in [89]).”
Asymptotically exact inference in differentiable generative models (2016, methodology): “Finding informative low-dimensional summaries is often critical to getting ABC methods to work well in practice and there is a wide literature on methods for choosing summary statistics - see [77] and [17] for reviews.”
Asymptotic properties of approximate Bayesian computation (2016, background): “Once again, εT {log(vT )}/vT v−1 T ensures that (6) will satisfy Assumption [A1].”
Bayesian computation: a summary of the current state, and samples backwards and forwards (2015, methodology): “This imposed reduction of the statistical information raises many relevant questions, from the choice of summary statistics (Blum et al. 2013) to the consistency of the ensuing inference (Robert et al. 2011).”
On the Asymptotic Efficiency of Approximate Bayesian Computation Estimators (2015, methodology): “s such as E[jY obs] suggested in [17, Theorem 3]. How to choose the dimension dof s obs is of interest, since larger dgives possibly more informative s obsbut slower convergence of hbwhen Nincreases [8]. The following proposition states that when dexceeds the dimension of the parameter, h ABCbased o”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, methodology): “…is the fact that virtually all algorithms rely on summary statistics in order to reduce the dimensionality of the data (Fearnhead & Prangle, 2012; Blum et al., 2013; Jiang et al., 2017)—designing summary statistics can require domain-speciﬁc knowledge and/or separate optimization procedures, and…”
Approximate Bayesian computation via regression density estimation (2012, methodology): “Blum et al. (2012), Fearnhead & Prangle (2012)). A step by step description of how to of construct O L.Sj / is: Step 1 Sample .S.i/, .i// L.Sj /h. / for i D 1, : : : , N. Define S.i/ D S.i/1, : : : , S.i/k . Step 2 For each Sj, fit the mixture model (3) to the marginal samples .S.i/j, .i//. Step 3 C”
Approximate Bayesian Computation (2013, methodology): “A review of methods for choosing summary statistics is available [29], which may provide valuable guidance in practice.”
ABC–CDE: Toward Approximate Bayesian Computation With Complex High-Dimensional Data and Limited Simulations (2018, methodology): “For a general review of dimension reduction methods for ABC, we refer the reader to Blum et al. (2013), who classified current approaches in three classes: (a) best subset selection approaches, (b) projection techniques, and (c) regularization techniques.”
Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network (2015, background): “measure of sufficiency (Joyce and Marjoram (2008)), entropy (Nunes and Balding (2010)), Akaike and Bayesian information criteria (Blum et al. (2013)), and the “best” subset is chosen to be the summary statistic. The second class is linear regression approach, which constructs summary statistics by l”
Likelihood-Free Inference by Ratio Estimation (2016, background): “…works on summary statistics selection in the framework of approximate Bayesian computation (Aeschbacher et al., 2012; Fearnhead and Prangle, 2012; Blum et al., 2013; Gutmann et al., 2014; Marin et al., 2016; Gutmann et al., 2018; Jiang et al., 2018), there is comparably little corresponding work…”
Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology (2018, methodology): “Data compression schemes should be carefully designed to reduce the data to the smallest set of summaries possible, whilst retaining as much information about the parameters of interest as possible (see Blum et al. 2013 for a review).”