Likelihood-free MCMC with Approximate Likelihood Ratios

By Joeri Hermans, Volodimir Begy, Gilles Louppe. 2019

21 citations recorded by the source corpus.

We propose a novel approach for posterior sampling with intractable likelihoods. This is an increasingly important problem in scientific applications where models are implemented as sophisticated computer simulations. As a result, tractable densities are not available, which forces practitioners to rely on approximations during inference. We address the intractability of densities by training a parameterized classifier whose output is used to approximate likelihood ratios between arbitrary model parameters. In turn, we are able to draw posterior samples by plugging this approximator into common Markov chain Monte Carlo samplers such as Metropolis-Hastings and Hamiltonian Monte Carlo. We demonstrate the proposed technique by fitting the generating parameters of implicit models, ranging from a linear probabilistic model to settings in high energy physics with high-dimensional observations. Finally, we discuss several diagnostics to assess the quality of the posterior.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

The frontier of simulation-based inference (2019, methodology): “In most cases, however, we recommend trying methods based on training a neural network surrogate for the likelihood (39, 63) or the likelihood ratio (51, 67, 71).”
sbi: A toolkit for simulation-based inference (2020, methodology): “…estimator of the likelihood, which then allows to sample from the posterior using e.g. MCMC. • Sequential Neural Ratio Estimation (SNRE) (Durkan et al., 2020; Hermans et al., 2020) trains a classifier to estimate density ratios, which in turn can be used to sample from the posterior e.g. with MCMC.”