Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows

By George Papamakarios, David C. Sterratt, Iain Murray. 2018

484 citations recorded by the source corpus.

We present Sequential Neural Likelihood (SNL), a new method for Bayesian inference in simulator models, where the likelihood is intractable but simulating data from the model is possible. SNL trains an autoregressive flow on simulated data in order to learn a model of the likelihood in the region of high posterior density. A sequential training procedure guides simulations and reduces simulation cost by orders of magnitude. We show that SNL is more robust, more accurate and requires less tuning than related neural-based methods, and we discuss diagnostics for assessing calibration, convergence and goodness-of-fit.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Statistics and Probability

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

The frontier of simulation-based inference (2019, neutral): “ST A TI ST IC S (33, 56) or the likelihood ratio (60, 62, 64).”
Neural Spline Flows (2019, background): “In recent years, normalizing flows have received widespread attention in the machine-learning literature, seeing successful use in density estimation [9, 41], variational inference [29, 35, 44, 55], image, audio and video generation [25, 27, 31, 43], likelihood-free inference [42], and learning maxi”
Normalizing Flows for Probabilistic Modeling and Inference (2019, methodology): “Alternatively, we can fit a flow-based model q(x |η) conditioned on η in order to approximate the intractable likelihood p(x |η) (Papamakarios et al., 2019).”
Likelihood-free MCMC with Approximate Likelihood Ratios (2019, methodology): “The summary statistic in ABC follows the deﬁnition from [8]: the mean of the population sizes and the autocorrelation of the population time series at different lags.”
Fast likelihood-free cosmology with neural density estimators and active learning (2019, methodology): “Density-estimation likelihood-free inference (DELFI; Bonassi et al. 2011; Fan et al. 2013; Papamakarios & Murray 2016; Lueckmann et al. 2017; Papamakarios et al. 2018; Lueckmann et al. 2018; Alsing et al. 2018b) aims to train a flexible density estimator for the target posterior from a set of…”
Nuisance hardened data compression for fast likelihood-free inference (2019, methodology): “Mixture density networks (MDN) and masked autoregressive flows (MAF) have proven useful conditional density estimators in this context (Papamakarios & Murray 2016; Papamakarios et al. 2018; Alsing et al. 2019).”
sbi: A toolkit for simulation-based inference (2020, background): “To aid in effective application of these algorithms to a wide range of problems, sbi closely integrates with PyTorch and offers state-of-the-art neural network-based SBI algorithms (Greenberg et al., 2019; Hermans et al., 2020; Papamakarios et al., 2019) with flexible choice of network architectures”
Bayesian Synthetic Likelihood (2017, methodology): “In contrast to the BSL posterior π B ( θ | S n ), if it were feasible to use the exact synthetic likelihood, we could obtain samples from the posterior density target which is referred to as the exact BSL posterior, while (2022) estimate the rate parameters of a stochastic kinetic model in genetics ”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, methodology): “A powerful recent approach, sequential neural likelihood (SNL), trains a neural conditional density estimator (e.g. masked ﬂow, Papamakarios et al., 2017) to estimate p ( x | θ ) for all x and θ (Papamakarios et al., 2018; Lueckmann et al., 2018).”
Mining for Dark Matter Substructure: Inferring Subhalo Population Properties from Strong Lenses with Machine Learning (2019, background): “…et al. 2017; Louppe & Cranmer 2017; Lueckmann et al. 2017; Papamakarios et al. 2017; Tran et al. 2017; Chen et al. 2018; Dinev & Gutmann 2018; Grathwohl et al. 2018; Huang et al. 2018; Kingma & Dhariwal 2018; Lueckmann et al. 2018; Papamakarios et al. 2018; Alsing et al. 2019; Hermans et al. 2019).”
Likelihood-Free Inference by Ratio Estimation (2016, methodology): “The models employed include regression models and neural networks, Gaussian processes as well as normalising ﬂows (Beaumont et al., 2002; Blum, 2010; Wilkinson, 2014; Gutmann and Corander, 2016; Papamakarios and Murray, 2016; Papamakarios et al., 2017, 2019; Chen and Gutmann, 2019).”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, methodology): “Contrary to our approach, SNL cannot directly provide estimates of the posterior posterior density function.”
Sequential Neural Methods for Likelihood-free Inference (2018, methodology): “[19], we used slice sampling, which performed robustly on each of our experiments.”
Unconstrained Monotonic Neural Networks (2019, background): “…ﬂows deﬁned as stacks of reversible transformations have proven to be quite efﬁcient for density estimation of empirical distributions [Papamakarios et al., 2019, 2017, Huang et al., 2018], as well as to improve posterior modeling in variational auto-encoders (VAE) [Germain et al.,…”