Debiasing Evidence Approximations: On Importance-weighted Autoencoders and Jackknife Variational Inference

By Sebastian Nowozin. 2018

64 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Residual Energy-Based Models for Text Generation (2020, methodology): “Since the log-likelihood required by PPL relies on estimating the partition function , we derive two estimators for the log-partition function log Z θ based on the work of Nowozin (2018).”
Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation (2019, background): “Recent work has investigated tighter objectives [6, 20, 18, 17, 23, 25], based on the following principle: Let R be an estimator of the likelihood—i.”
Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives (2018, methodology): “In particular, we show that this estimator reduces the variance of the IWAE gradient, the reweighted wake-sleep update (RWS) (Bornschein & Bengio, 2014), and the jackknife variational inference (JVI) gradient (Nowozin, 2018).”
Residual Energy-Based Models for Text (2021, methodology): “Since the log-likelihood required by PPL relies on estimating the partition function , we derive two estimators for the log-partition function log Z θ based on the work of Nowozin (2018).”
Energy-Based Models for Text (2020, methodology): “Since the log-likelihood required by PPL relies on estimating the partition function Zθ = ∑ x Pφ(x) exp(−Eθ(x)) = Ex∼Pφ exp(−Eθ(x)), we derive two estimators for the logpartition function logZθ based on the work of Nowozin (2018).”
Appendix for: How Good is the Bayes Posterior in Deep Neural Networks Really? (2020, unknown): “In particular, expectations of smooth functions of empirical means of S samples have an expansion of the form, (Nowozin, 2018; Schucany et al., 1971), E[C(S)] = C(∞) + a1 1 S + a2 1 S2 + . . . . (24) Risk Asymptotics Experiment: if we can estimate C(∞) we know what performance we could achieve if…”
MIWAE: Deep Generative Modelling and Imputation of Incomplete Data Sets (2019, methodology): “As a result, the properties of the MIWAE bound described in this subsection are directly obtained by leveraging recent theoretical work on the IWAE bound (Cremer et al., 2017; Nowozin, 2018; Rainforth et al., 2018; Domke & Sheldon, 2018; Tucker et al., 2019).”
Importance Weighted Hierarchical Variational Inference (2019, background): “Nowozin (2018) has shown that the standard IWAE can be seen as a biased estimate of the log marginal likelihood with the bias of order O(1/M).”