Approximate Bayesian computation via regression density estimation

By Yanan Fan, David J. Nott, Scott Anthony Sisson. 2012

63 citations recorded by the source corpus.

Approximate Bayesian computation (ABC) methods, which are applicable when the likelihood is difficult or impossible to calculate, are an active topic of current research. Most current ABC algorithms directly approximate the posterior distribution, but an alternative, less common strategy is to approximate the likelihood function. This has several advantages. First, in some problems, it is easier to approximate the likelihood than to approximate the posterior. Second, an approximation to the likelihood allows reference analyses to be constructed based solely on the likelihood. Third, it is straightforward to perform sensitivity analyses for several different choices of prior once an approximation to the likelihood is constructed, which needs to be done only once. The contribution of the present paper is to consider regression density estimation techniques to approximate the likelihood in the ABC setting. Our likelihood approximations build on recently developed marginal adaptation density estimators by extending them for conditional density estimation. Our approach facilitates reference Bayesian inference, as well as frequentist inference. The method is demonstrated via a challenging problem of inference for stereological extremes, where we perform both frequentist and Bayesian inference. Copyright © 2013 John Wiley & Sons Ltd

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Masked Autoregressive Flow for Density Estimation (2017, neutral): “In simulation-based likelihood-free inference, conditional density estimators can learn models for the likelihood [5] or the posterior [23] from simulated data.”
Mining gold from implicit models to improve likelihood-free inference (2018, background): “6), summing over x ∈ [5, 15], versus the training sample size (which refers to the total number of x samples, distributed over 10 values of θ ∈ [−1,−0.”
Efficient Acquisition Rules for Model-Based Approximate Bayesian Computation (2017, background): “Some other alternative approaches are considered by Fan et al. (2013); Papamakarios and Murray (2016).”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, methodology): “Approaches based on estimating the likelihood (Wood, 2010; Fan et al., 2013; Turner & Sederberg, 2014; Papamakar-ios et al., 2018; Lueckmann et al., 2018) or posterior (Le et al., 2017; Papamakarios & Murray, 2016; Lueckmann et al., 2017; Chan et al., 2018) (which can be traced back to…”
A simple approach to maximum intractable likelihood estimation (2013, background): “The quality of the approximation to the posterior and theoretical properties of the estimators obtained with ABC have been studied in Wilkinson (2008); Blum (2010); Marin et al. (2011); Dean et al. (2011); and Fearnhead and Prangle (2012).”
Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimation (2017, background): “Other ﬁelds where conditional density estimation plays a key role are time series forecasting in economics (Kalda and Siddiqui, 2013) and approximate Bayesian methods (Fan et al., 2013; Izbicki et al., 2014; Papamakarios and Murray, 2016).”
Likelihood-Free Inference by Ratio Estimation (2016, methodology): “Ease of use is shared with existing likelihood-free inference methods that are based on a Gaussianity assumption of the summary statistics or the data generating process (Wood, 2010; Fan et al., 2013; Price et al., 2016).”
Mining for Dark Matter Substructure: Inferring Subhalo Population Properties from Strong Lenses with Machine Learning (2019, methodology): “…large number of methods have been developed that train neural networks to estimate the likelihood function, likelihood ratio function, or posterior(Fan et al. 2012; Dinh et al. 2014, 2016; Cranmer et al. 2015; Germain et al. 2015; Jimenez Rezende & Mohamed 2015; Paige & Wood 2016; Papamakarios &…”
Flexible statistical inference for mechanistic models of neural dynamics (2017, methodology): “Synthetic-likelihood methods [28, 21, 29] approximate the likelihood p(x|θ) using multivariate Gaussians fitted to repeated simulations given θ (see [30, 31] for generalisations).”
Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation (2016, methodology): “Other parametric approaches try to approximate the likelihood p(x |θ), either with simple Gaussian models [34] or mixture models [8].”
ABC–CDE: Toward Approximate Bayesian Computation With Complex High-Dimensional Data and Limited Simulations (2018, background): “…works in this direction are Papamakarios and Murray (2016) and Lueckmann et al. (2017), which are based on conditional neural density estimation, Fan, Nott, and Sisson (2013) and © 2019 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North…”
Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology (2018, methodology): “With the two-step compression scheme defined, we then introduce Density Estimation Likelihood-Free Inference (delfi; Bonassi et al. 2011; Fan et al. 2013; Papamakarios & Murray 2016), which learns a parameterized model for the joint density of the parameters and compressed statistics P(θ, t), from…”
Hamiltonian ABC (2015, methodology): “To perform inference in likelihood-free models, a broad class of algorithms called Approximate Bayesian Computation (Beaumont et al., 2002; Marjoram et al., 2003; Sisson et al., 2007; Sisson & Fan, 2010; Marin et al., 2012; Fan et al., 2013) are employed.”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, background): “Other predecessors to SNL include approximating the likelihood as a linear-Gaussian model [39], or as a mixture of Gaussian copulas with marginals modelled by mixtures of experts [21].”