Asymptotically exact inference in differentiable generative models

By Matthew M. Graham, Amos J. Storkey. 2016

38 citations recorded by the source corpus.

Many generative models can be expressed as a differentiable function of random inputs drawn from some simple probability density. This framework includes both deep generative architectures such as Variational Autoencoders and a large class of procedurally defined simulator models. We present a method for performing efficient MCMC inference in such models when conditioning on observations of the model output. For some models this offers an asymptotically exact inference method where Approximate Bayesian Computation might otherwise be employed. We use the intuition that inference corresponds to integrating a density across the manifold corresponding to the set of inputs consistent with the observed outputs. This motivates the use of a constrained variant of Hamiltonian Monte Carlo which leverages the smooth geometry of the manifold to coherently move between inputs exactly consistent with observations. We validate the method by performing inference tasks in a diverse set of models.

Subfields: Artificial Intelligence, Statistics and Probability, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Generative Models

Citations in the corpus

The frontier of simulation-based inference (2019, background): “Powerful algorithms are available for the case with differentiable simulators (43), for simulators for which the joint likelihood of data and latent variables is accessible (44), and for simulators explicitly written as a probabilistic model in a probabilistic programming framework (78).”
CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training (2017, background): “See [29, 14] for other works on conditioning for implicit generative models.”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, background): “Advanced ABC algorithms that work for (cid:15) = 0 exist [27], but require the simulator to be diﬀerentiable.”