Mining gold from implicit models to improve likelihood-free inference

By J. Brehmer, Gilles Louppe, Juan Pavez, Kyle Cranmer. 2018

207 citations recorded by the source corpus.

Significance In scientific fields as diverse as particle physics, genetics, and epidemiology, complex computer simulations provide the most accurate description of phenomena, but the corresponding likelihood function cannot be computed. This is a major challenge for statistical inference. We present inference techniques for this case that combine the insight that additional latent information can be extracted from the simulator with the power of neural networks in regression and density estimation tasks. In toy problems and a real-life particle physics problem, these techniques lead to better sample efficiency than established methods, demonstrating the potential to enable more precise scientific results from large-scale experiments across many disciplines. Simulators often provide the best description of real-world phenomena. However, the probability density that they implicitly define is often intractable, leading to challenging inverse problems for inference. Recently, a number of techniques have been introduced in which a surrogate for the intractable density is learned, including normalizing flows and density ratio estimators. We show that additional information that characterizes the latent process can often be extracted from simulators and used to augment the training data for these surrogate models. We introduce several loss functions that leverage these augmented data and demonstrate that these techniques can improve sample efficiency and quality of inference.

Subfields: Artificial Intelligence, Control and Systems Engineering, Mechanical Engineering, Probabilistic & Bayesian Methods, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Mechanical Engineering

In Probabilistic & Bayesian Methods

In AI for Physical Sciences

Citations in the corpus

Likelihood-free MCMC with Approximate Likelihood Ratios (2019, background): “However, recent works [57, 58] make it possible to signiﬁcantly reduce the cost of this step, provided joint likelihood ratios and scores can be extracted from the simulator.”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, background): “This scenario is common in neuroscience (Paninski & Cunningham, 2017), physics (Brehmer et al., 2018), systems biology (Clarke et al., 2008) and inverse graphics (Romaszko et al., 2017).”
Likelihood-free inference with an improved cross-entropy estimator (2018, methodology): “[24–26] coined the term “mining gold” for the process of extracting the joint likelihood ratio and joint score from the simulator – while the augmented data require some effort to extract, they are extremely valuable.”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, background): “Rather than learning the likelihood p ( x | θ ), an alternative approach is learning the likelihood ratio p ( x | θ 1 ) /p ( x | θ 2 ) [11, 12, 15, 30, 61, 74], or the likelihood-to-marginal ratio p ( x | θ ) /p ( x ) [19, 33, 77].”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, background): “The most powerful test-statistic to compare two hypotheses θ 0 and θ 1 for an observation x is the likelihood ratio (J. Ney-man, 1933) thereby obtaining the likelihood ratio as In the literature, this is known as the likelihood ratio trick ( LRT ) (Cranmer et al., 2015; Mohamed & Lakshmi-narayanan, ”
INFERNO: Inference-Aware Neural Optimisation (2018, background): “his work can also be extended, as done by Baldi et al. [19] by a subset of the inference parameters to obtain a parametrised family of summary statistics with a single model. Recently, Brehmer et al. [20, 21, 22] further extended the approach of parametrised classiﬁers to better exploit the latent-s”
Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale (2019, background): “Inference in simulator-based models remains a challenging problem with potential impact across many disciplines [14, 60].”