Generalized massive optimal data compression

By Justin Alsing, Benjamin Dan Wandelt. 2017

90 citations recorded by the source corpus.

Data compression has become one of the cornerstones of modern astronomical data analysis, with the vast majority of analyses compressing large raw datasets down to a manageable number of informative summaries. In this paper we provide a general procedure for optimally compressing $N$ data down to $n$ summary statistics, where $n$ is equal to the number of parameters of interest. We show that compression to the score function -- the gradient of the log-likelihood with respect to the parameters -- yields $n$ compressed statistics that are optimal in the sense that they preserve the Fisher information content of the data. Our method generalizes earlier work on linear Karhunen-Lo\'{e}ve compression for Gaussian data whilst recovering both lossless linear compression and quadratic estimation as special cases when they are optimal. We give a unified treatment that also includes the general non-Gaussian case as long as mild regularity conditions are satisfied, producing optimal non-linear summary statistics when appropriate. As a worked example, we derive explicitly the $n$ optimal compressed statistics for Gaussian data in the general case where both the mean and covariance depend on the parameters.

Subfields: Artificial Intelligence, Statistics and Probability, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In AI for Physical Sciences

Citations in the corpus

The frontier of simulation-based inference (2019, background): “Just like the likelihood, the score itself is generally intractable, but it can be estimated based on quantity 5 and an exponential family approximation (71, 72).”
Mining gold from implicit models to improve likelihood-free inference (2018, background): “For inference in a sufﬁciently small neighborhood around a reference point θ ref , a precise estimator of the score ˆ t ( x | θ ref ) therefore deﬁnes a vector of ideal summary statistics that contain all the information in an observation x on the parameters θ [see also 3, 46, 47].”
The Quijote Simulations (2019, background): “(7) This is because this term is expected to be small (Kodwani et al. 2019) but including it will also lead to an underestimation of the parameter errors (Carron 2013; Alsing & Wandelt 2018).”
Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology (2018, methodology): “…Fisher information, to the extent that the assumed form for the (unknown or intractable) likelihood is a good approximation to the true likelihood function and the fiducial expansion point is close to the maximum-likelihood (this can be iterated if necessary; see Alsing & Wandelt 2018 for details).”
Nuisance hardened data compression for fast likelihood-free inference (2019, background): “When the likelihood is known, the score function – the gradient of the log-likelihood with respect to the parameters – yields asymptotically optimal compression of N data down to p summaries (Alsing & Wandelt 2018).”