Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation

By George Papamakarios, Iain Murray. 2016

171 citations recorded by the source corpus.

Many statistical models can be simulated forwards but have intractable likelihoods. Approximate Bayesian Computation (ABC) methods are used to infer properties of these models from data. Traditionally these methods approximate the posterior over parameters by conditioning on data being inside an $\epsilon$-ball around the observed data, which is only correct in the limit $\epsilon\!\rightarrow\!0$. Monte Carlo methods can then draw samples from the approximate posterior to approximate predictions or error bars on parameters. These algorithms critically slow down as $\epsilon\!\rightarrow\!0$, and in practice draw samples from a broader distribution than the posterior. We propose a new approach to likelihood-free inference based on Bayesian conditional density estimation. Preliminary inferences based on limited simulation data are used to guide later simulations. In some cases, learning an accurate parametric representation of the entire true posterior distribution requires fewer model simulations than Monte Carlo ABC methods need to produce a single sample from an approximate posterior.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Sequential Neural Methods for Likelihood-free Inference (2018, methodology): “Again, we follow the experimental setup of Papamakarios and Murray [17].”
Conditional Density Estimation with Bayesian Normalising Flows (2018, methodology): “Papamakarios & Murray (2016) perform stochastic variational inference in a mixture density network for an application to approximately Bayesian computation and Depeweg et al. (2017) and coauthors use stochastic inputs to a Bayesian neural network for an application to reinforcement learning.”
Likelihood-free inference with emulator networks (2018, background): “Approximate Bayesian Computation (ABC) aims to close this gap by providing Bayesian inference algorithms that can be used with simulator models without requiring explicit likelihoods [3, 4, 5, 6, 7].”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, methodology): “In our experiments we follow the setup of Papamakarios and Murray [59], where x is 9 features of the population timeseries.”
Dynamic Likelihood-free Inference via Ratio Estimation (DIRE) (2018, methodology): “Neural networks have been used before in ABC: Blum and Francois (2010); Papamakarios and Murray (2016) used them in the context of regression ABC where the summary statistics are assumed to be given, and, more relevant for the topic of this paper, Jiang et al. (2018) used them to learn summary…”
Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimation (2017, background): “Other ﬁelds where conditional density estimation plays a key role are time series forecasting in economics (Kalda and Siddiqui, 2013) and approximate Bayesian methods (Fan et al., 2013; Izbicki et al., 2014; Papamakarios and Murray, 2016).”
Dynamical Sampling with Langevin Normalization Flows (2019, unknown): “Recently, neural density estimators are widely used in the approximation of data distributions [32,33] and variational inference [27].”
Asymptotically exact inference in differentiable generative models (2016, methodology): “We used the nine summary statistics employed in a similar Lotka–Volterra inference problem in [71] - the means and log variances of the two sequences, lag-one and lag-two autocorrelation coefficients and cross-correlation coefficient of the sequences.”
Flexible statistical inference for mechanistic models of neural dynamics (2017, methodology): “sted in the numerical stability of the method (‘robustness’) and the ‘ﬂexibility’ to approximate multi-modal posteriors. To illustrate the robustness of SNPE, we apply SNPE and the method proposed by [19] (which we refer to by Conditional Density Estimation for Likelihood-free Inference, CDE-LFI) to”
Likelihood-Free Inference by Ratio Estimation (2016, methodology): “The models employed include regression models and neural networks, Gaussian processes as well as normalising flows (Beaumont et al., 2002; Blum, 2010; Wilkinson, 2014; Gutmann and Corander, 2016; Papamakarios and Murray, 2016; Papamakarios et al., 2017, 2019; Chen and Gutmann, 2019).”