Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology

By Justin Alsing, Benjamin Dan Wandelt, S. Feeney. 2018

155 citations recorded by the source corpus.

Many statistical models in cosmology can be simulated forwards but have intractable likelihood functions. Likelihood-free inference methods allow us to perform Bayesian inference from these models using only forward simulations, free from any likelihood assumptions or approximations. Likelihood-free inference generically involves simulating mock data and comparing to the observed data; this comparison in data space suffers from the curse of dimensionality and requires compression of the data to a small number of summary statistics to be tractable. In this paper, we use massive asymptotically optimal data compression to reduce the dimensionality of the data space to just one number per parameter, providing a natural and optimal framework for summary statistic choice for likelihood-free inference. Secondly, we present the first cosmological application of Density Estimation Likelihood-Free Inference (delfi), which learns a parametrized model for joint distribution of data and parameters, yielding both the parameter posterior and the model evidence. This approach is conceptually simple, requires less tuning than traditional Approximate Bayesian Computation approaches to likelihood-free inference and can give high-fidelity posteriors from orders of magnitude fewer forward simulations. As an additional bonus, it enables parameter inference and Bayesian model comparison simultaneously. We demonstrate delfi with massive data compression on an analysis of the joint light-curve analysis supernova data, as a simple validation case study. We show that high-fidelity posterior inference is possible for full-scale cosmological data analyses with as few as ∼10^4 simulations, with substantial scope for further improvement, demonstrating the scalability of likelihood-free inference to large and complex cosmological data sets.

Subfields: Artificial Intelligence, Astronomy and Astrophysics, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Astronomy and Astrophysics

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

The frontier of simulation-based inference (2019, background): “Just like the likelihood, the score itself is generally intractable, but it can be estimated based on quantity 5 and an exponential family approximation (71, 72).”
Normalizing Flows for Probabilistic Modeling and Inference (2019, methodology): “Such simulator-based models are common in scientific fields such as cosmology (Alsing et al., 2018), high-energy physics (Brehmer et al.”
Likelihood-free inference with an improved cross-entropy estimator (2018, unknown): “t(x, z|θ0) = ∇θ log p(x, z|θ) ∣∣∣∣ θ0 , (3)”
Mining gold from implicit models to improve likelihood-free inference (2018, background): “For inference in a sufﬁciently small neighborhood around a reference point θ ref , a precise estimator of the score ˆ t ( x | θ ref ) therefore deﬁnes a vector of ideal summary statistics that contain all the information in an observation x on the parameters θ [see also 3, 46, 47].”
Generalized massive optimal data compression (2017, background): “…2012; Cameron & Pettitt 2012; Weyant et al. 2013; Robin et al. 2014; Lin & Kilbinger 2015; Akeret et al. 2015; Ishida et al. ⋆ E-mail: jalsing@flatironinstitute.org 2015; Jennings et al. 2016; Hahn et al. 2017; Kacprzak et al. 2017; Carassou et al. 2017; Davies et al. 2017; Alsing et al. 2018).”
Nuisance hardened data compression for fast likelihood-free inference (2019, methodology): “…networks have two hidden layers with 5× as many units as parameters to be inferred, and simulations are run in batches of 50× the number of parameters in the 7 This proposal density is inspired by the optimal parameter proposal for Approximate Bayesian Computation (Alsing et al. 2018a). problem.”
Fast likelihood-free cosmology with neural density estimators and active learning (2019, methodology): “…function is known, the score function t = ∇θ ln p(d|θ) yields compression of N data down to p summaries, one per parameter, such that the Fisher information of the data is preserved (provided the gradient is taken close to the true parameters, Alsing & Wandelt 2018b; Alsing et al. 2018b).”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, background): “…themselves naturally to modelling mechanistic processes such as particle collisions in high energy physics [1, 71], universe evolution in cosmology [2, 67], stochastic dynamical systems in ecology and evolutionary biology [64, 65, 84, 85], macroeconomic models in econometrics [3, 26], and…”