Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference

By Geoffrey Roeder, Yuhuai Wu, David Kristjanson Duvenaud. 2017

225 citations recorded by the source corpus.

We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient estimator whose variance approaches zero as the approximate posterior approaches the exact posterior. We analyze the behavior of this gradient estimator theoretically and empirically, and generalize it to more complex variational distributions such as mixtures and importance-weighted posteriors.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

NVAE: A Deep Hierarchical Variational Autoencoder (2020, background): “…as reducing the gap between approximate and true posterior distributions [3, 4, 5, 6, 7, 8, 9, 10], formulating tighter bounds [11, 12, 13, 14], reducing the gradient noise [15, 16], extending VAEs to discrete variables [17, 18, 19, 20, 21, 22, 23], or tackling posterior collapse [24, 25, 26, 27].”
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models (2017, methodology): “Because the reparameterization gradient only enters the REBAR estimator through differences of reparameterization gradients, we implicitly implement the recommendation from (Roeder et al., 2017 The standard error of the mean is given in the Appendix.”
DreamFusion: Text-to-3D using 2D Diffusion (2022, background): “(14) Sticking-the-Landing (Roeder et al., 2017) shows that keeping the path derivative gradient while discarding the score function gradient can lead to reduced variance as the path derivative term can be correlated with other terms in the loss.”
BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery (2021, background): “possible to compute the term Ez∼qφ [log qφ(z)− log p(z)] = DKL [qφ, p] in closed form, possibly reducing variance compared to Monte Carlo estimates [43].”
MIWAE: Deep Generative Modelling and Imputation of Incomplete Data Sets (2019, unknown): “While this was the original approach of Burda et al. (2016), Roeder et al. (2017) and Rainforth et al. (2018) noticed both experimentally and theoretically that such gradients may have an overly large variance.”
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models (2019, background): “To see this, we can expand (6) as: Roeder et al. (2017) ﬁnd that when K > 1 , the ﬁrst term within paranthesis in (7) need not be zero even when the approximate posterior matches true posterior everywhere, which can contribute to signiﬁcant variance in the gradient estimator.”
Variance reduction properties of the reparameterization trick (2018, methodology): “There are numerous ways to represent this gradient, each one giving a speciﬁc estimator: see Roeder et al. (2017) for some choices.”
Undirected Graphical Models as Approximate Posteriors (2019, methodology): “Note that Roeder et al. (2017) and Tucker et al. (2018) do not introduce any bias to the gradient and are known to reduce variance.”
Augment and Reduce: Stochastic Inference for Large Categorical Distributions (2018, background): “Even when it is, the Monte Carlo estimator may have lower variance (Roeder et al., 2017).”
Functional Variational Bayesian Neural Networks (2019, background): “) But note that it may be useful as a control variate (Roeder et al., 2017).”
Direct Optimization through arg max for Discrete Variational Auto-Encoder (2019, methodology): “As a result, our methods may beneﬁt from the REBAR/RELAX framework, which directs biased gradients towards the unbiased gradient [33, 28].”
An Introduction to Variational Autoencoders (2019, methodology): “The gradient variance can in some cases be reduced by ’carving up the ELBO’ (Hoffman and Johnson, 2016; Roeder et al., 2017) and using a modified gradient estimator.”