A Universal Marginalizer for Amortized Inference in Generative Models

By Laura Douglas, Iliyan Zarov, Kostis Gourgoulias, Chris Lucas, Chris Hart, Adam Baker, Maneesh Sahani, Yura N. Perov, Saurabh Johri. 2017

31 citations recorded by the source corpus.

We consider the problem of inference in a causal generative model where the set of available observations differs between data instances. We show how combining samples drawn from the graphical model with an appropriate masking function makes it possible to train a single neural network to approximate all the corresponding conditional marginal distributions and thus amortize the cost of inference. We further demonstrate that the efficiency of importance sampling may be improved by basing proposals on the output of the neural network. We also outline how the same network can be used to generate samples from an approximate joint posterior via a chain decomposition of the graph.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learning Neural Causal Models from Unknown Interventions (2019, methodology): “Neural network methods equipped with learned masks, such as [46, 47, 48, 49], exist in the literature, but only a few [44] have been adapted to causal inference.”
Variational Autoencoder with Arbitrary Conditioning (2018, methodology): “And ﬁnally, we evaluate VAEAC against the competing method called Universal Marginalizer (Douglas et al., 2017).”
Understanding Uncertainty in Bayesian Neural Networks (2019, background): “Unfortunately, unlike other DGMs (Douglas et al., 2017; Germain et al., 2015; Iizuka et al., 2017), the stock VAE framework provides no straight forward way to perform conditional generation.”
A comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis (2018, background): “Given a set of userentered presenting symptoms and risk-factors, the model infers the most likely conditions and generates follow-up questions [4, 5, 6, 7].”
Flow Models for Arbitrary Conditional Likelihoods (2019, background): “Previous attempts to learn probability distributions conditioned on arbitrary subsets of known covariates include the Universal Marginalizer (Douglas et al. 2017), which is trained as a feed-forward network to approximate the marginal posterior distribution of each unobserved dimen-sion conditioned…”
MultiVerse: Causal Reasoning using Importance Sampling in Probabilistic Programming (2019, methodology): “, 2017) and in particular importance sampling inference (Douglas et al., 2017; Walecki et al., 2018) can be used to facilitate the posterior inference problem.”
Explaining by Removing: A Unified Framework for Model Explanation (2020, methodology): “Frye et al. (2020) demonstrated a similar approach using a variational autoencoder-like model (Ivanov et al., 2018), and future work could leverage other conditional generative models (Douglas et al., 2017; Belghazi et al., 2019).”