Likelihood-free inference via classification

By Michael U Gutmann, Ritabrata Dutta, Samuel Kaski, Jukka Corander. 2014

136 citations recorded by the source corpus.

Increasingly complex generative models are being used across disciplines as they allow for realistic characterization of data, but a common difficulty with them is the prohibitively large computational cost to evaluate the likelihood function and thus to perform likelihood-based statistical inference. A likelihood-free inference framework has emerged where the parameters are identified by finding values that yield simulated data resembling the observed data. While widely applicable, a major difficulty in this framework is how to measure the discrepancy between the simulated and observed data. Transforming the original problem into a problem of classifying the data into simulated versus observed, we find that classification accuracy can be used to assess the discrepancy. The complete arsenal of classification methods becomes thereby available for inference of intractable generative models. We validate our approach using theory and simulations for both point estimation and Bayesian inference, and demonstrate its use on real data by inferring an individual-based epidemiological model for bacterial infections in child care centers.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA (2016, background): “Methods such as noise-contrastive estimation [11] and generative adversarial nets [8], see also [10], are similar in spirit, but clearly distinct from TCL which uses the temporal structure of the data by contrasting different time segments.”
Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning (2018, methodology): “This unifies and generalizes the methods by Harmeling et al. (2003); Sprekeler et al. (2014); Hyvärinen and Morioka (2017b,a), giving a general framework where it is not necessary to specifically have a temporal (or even spatial) structure in the data.”
Learning in Implicit Generative Models (2016, background): “The other signiﬁcant body of research that focusses on learning in implicit generative models using the lens of moment matching is approximate Bayesian computation (ABC) (Marin et al., 2012).”
Dynamic Likelihood-free Inference via Ratio Estimation (DIRE) (2018, background): “The exact posterior can be computed numerically to high accuracy (see e.g. Gutmann et al. 2018, Supplementary Material 1.2.4).”
On Fenchel Mini-Max Learning (2019, background): “1 Likelihood-free modeling & latent variable models One can reformulate generative adversarial networks (GANs) [25, 30] into a latent-variable model, by introducing arbitrarily small Gaussian perturbations.”
Nonlinear ICA of Temporally Dependent Stationary Sources (2017, methodology): “This leads to a simple method which needs hardly any new algorithmic developments, since it is based on formulating the unsupervised learning problem in terms of logistic regression, in the spirit of noise-contrastive estimation (Gutmann and Hyvärinen, 2012), timecontrastive learning (Hyvärinen and ”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, background): “The most powerful test-statistic to compare two hypotheses θ 0 and θ 1 for an observation x is the likelihood ratio (J. Ney-man, 1933) thereby obtaining the likelihood ratio as In the literature, this is known as the likelihood ratio trick ( LRT ) (Cranmer et al., 2015; Mohamed & Lakshmi-narayanan, ”
Mining gold from implicit models to improve likelihood-free inference (2018, methodology): “Most established inference methods, such as ABC and techniques based on neural density estimation, only use samples of observables from the simulator.”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, background): “Rather than learning the likelihood p(x |θ), an alternative approach is learning the likelihood ratio p(x |θ1)/p(x |θ2) [11, 12, 15, 30, 61, 74], or the likelihood-to-marginal ratio p(x |θ)/p(x) [19, 33, 77].”
Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models (2015, background): “The results from that literature are, however, not straightforwardly applicable to our work: We may only be given a certain discrepancy measure ∆θ without underlying summary statistics Φθ (Gutmann et al., 2014).”
Diverse Image Synthesis From Semantic Layouts via Conditional IMLE (2018, methodology): “Most modern image synthesis methods are based on generative adversarial nets (GANs) [11, 12].”
Mining for Dark Matter Substructure: Inferring Subhalo Population Properties from Strong Lenses with Machine Learning (2019, background): “…2016; Uria et al. 2016; van den Oord et al. 2016b,c,a; Tran et al. 2017; Pa-pamakarios et al. 2017; Louppe & Cranmer 2017; Lueck-mann et al. 2017; Gutmann et al. 2017; Chen et al. 2018; Dinev & Gutmann 2018; Grathwohl et al. 2018; Huang et al. 2018; Kingma & Dhariwal 2018; Lueck-mann et al.…”
Conditional Noise-Contrastive Estimation of Unnormalised Models (2018, unknown): “…adapt the conditional noise distribution to make the classiﬁcation task successively more challenging, as it was done in some simulations for NCE (Gutmann & Hyv¨arinen, 2010), and generally for learning in generative latent variable models (Gutmann et al., 2014; Goodfellow et al., 2014).”
Variational Generative Stochastic Networks with Collaborative Shaping (2015, background): “…parameters of g θ so as to match some of the statistical moments of G to the corresponding moments observed in an empirical sample from D , and more recent approaches based on minimizing the ability of some classiﬁer to distinguish between G and D (Gutmann et al., 2014a;b; Goodfellow et al., 2014).”
Likelihood-free MCMC with Approximate Likelihood Ratios (2019, background): “On the other hand, approaches such as [36] remove the need for handcrafted summary statistics.”