Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives

By George Tucker, Dieterich Lawson, Shixiang Shane Gu, Chris J. Maddison. 2018

115 citations recorded by the source corpus.

Deep latent variable models have become a popular model choice due to the scalable learning algorithms introduced by (Kingma & Welling, 2013; Rezende et al., 2014). These approaches maximize a variational lower bound on the intractable log likelihood of the observed data. Burda et al. (2015) introduced a multi-sample variational bound, IWAE, that is at least as tight as the standard variational lower bound and becomes increasingly tight as the number of samples increases. Counterintuitively, the typical inference network gradient estimator for the IWAE bound performs poorly as the number of samples increases (Rainforth et al., 2018; Le et al., 2018). Roeder et al. (2017) propose an improved gradient estimator, however, are unable to show it is unbiased. We show that it is in fact biased and that the bias can be estimated efficiently with a second application of the reparameterization trick. The doubly reparameterized gradient (DReG) estimator does not suffer as the number of samples increases, resolving the previously raised issues. The same idea can be used to improve many recently introduced training techniques for latent variable models. In particular, we show that this estimator reduces the variance of the IWAE gradient, the reweighted wake-sleep update (RWS) (Bornschein & Bengio, 2014), and the jackknife variational inference (JVI) gradient (Nowozin, 2018). Finally, we show that this computationally efficient, unbiased drop-in gradient estimator translates to improved performance for all three objectives on several modeling tasks.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

NVAE: A Deep Hierarchical Variational Autoencoder (2020, background): “…as reducing the gap between approximate and true posterior distributions [3, 4, 5, 6, 7, 8, 9, 10], formulating tighter bounds [11, 12, 13, 14], reducing the gradient noise [15, 16], extending VAEs to discrete variables [17, 18, 19, 20, 21, 22, 23], or tackling posterior collapse [24, 25, 26, 27].”
Variational Autoencoders and Nonlinear ICA: A Unifying Framework (2019, background): “The framework of variational autoencoders [18, 24] (VAEs) and its extensions (e.g. [3, 17, 26, 20]) offers a scalable set of techniques for learning deep latent-variable models and corresponding inference models.”
The Thermodynamic Variational Objective (2019, background): “While this problem can be ameliorated by reducing the variance of the gradient estimator in the case of reparameterizable latent variables [17], resolving it in the case of non-reparameterizable latent variables currently involves alternating optimization of model and inference networks [18–20].”
Undirected Graphical Models as Approximate Posteriors (2019, methodology): “…approaches, the number of f evaluations increases with t. can be ignored in VAEs as Eq[@ log q (z |x)] = 0 (Roeder et al., 2017), and it can be removed in importance weighted autoencoders using the doubly reparameterized gradient estimation (Tucker et al., 2018) (see Appendix C for more details).”
Importance Weighted Hierarchical Variational Inference (2019, methodology): “Notably, the IWHVI achieves much higher values of the Mutual Information than the SIVI, and the improved gradient estimator IWHVI-DReG enables even better results due to better auxiliary distribution τη(ψ|z, x) (Tucker et al., 2019).”
MIWAE: Deep Generative Modelling and Imputation of Incomplete Data Sets (2019, methodology): “A solution to this problem, based on a double use of the reparametrisation trick, was recently proposed by Tucker et al. (2019).”
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models (2019, methodology): “To alleviate this, Tucker et al. (2019) re-apply the reparametrisation trick on it, yielding a doubly reparametrised gradient estimator (DReG): We implement the estimator speciﬁed in (8) when performing gradient updates for any experiment involving the IWAE objective.”