Likelihood-Free Inference by Ratio Estimation

By Owen Thomas, Ritabrata Dutta, Jukka Corander, Samuel Kaski, Michael U Gutmann. 2016

180 citations recorded by the source corpus.

We consider the problem of parametric statistical inference when likelihood computations are prohibitively expensive but sampling from the model is possible. Several so-called likelihood-free methods have been developed to perform inference in the absence of a likelihood function. The popular synthetic likelihood approach infers the parameters by modelling summary statistics of the data by a Gaussian probability distribution. In another popular approach called approximate Bayesian computation, the inference is performed by identifying parameter values for which the summary statistics of the simulated data are close to those of the observed data. Synthetic likelihood is easier to use as no measure of `closeness' is required but the Gaussianity assumption is often limiting. Moreover, both approaches require judiciously chosen summary statistics. We here present an alternative inference approach that is as easy to use as synthetic likelihood but not as restricted in its assumptions, and that, in a natural way, enables automatic selection of relevant summary statistic from a large set of candidates. The basic idea is to frame the problem of estimating the posterior as a problem of estimating the ratio between the data generating distribution and the marginal distribution. This problem can be solved by logistic regression, and including regularising penalty terms enables automatic selection of the summary statistics relevant to the inference task. We illustrate the general theory on canonical examples and employ it to perform inference for challenging stochastic nonlinear dynamical systems and high-dimensional summary statistics.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Likelihood-free MCMC with Approximate Likelihood Ratios (2019, methodology): “Contrary to other approaches which use a classiﬁer to directly approximate the likelihood-to-evidence ratio [6], ours by construction has the desirable property that log ˆ r ( x , θ 0 , θ 1 ) = − log ˆ r ( x , θ 1 , θ 0 ) .”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, methodology): “The most powerful test-statistic to compare two hypotheses θ 0 and θ 1 for an observation x is the likelihood ratio (J. Ney-man, 1933) thereby obtaining the likelihood ratio as In the literature, this is known as the likelihood ratio trick ( LRT ) (Cranmer et al., 2015; Mohamed & Lakshmi-narayanan, ”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, background): “Rather than learning the likelihood p ( x | θ ), an alternative approach is learning the likelihood ratio p ( x | θ 1 ) /p ( x | θ 2 ) [11, 12, 15, 30, 61, 74], or the likelihood-to-marginal ratio p ( x | θ ) /p ( x ) [19, 33, 77].”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, methodology): “Several recent studies investigated the use of (neural) classiﬁers for hypothesis testing or estimation of likelihood ratios: (Dutta et al., 2016; Gutmann et al., 2018) train binary classiﬁers to discriminate between the conditional and the marginal distribution, (Cran-mer et al., 2015) trains…”