The Generalized Reparameterization Gradient

By Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei. 2016

174 citations recorded by the source corpus.

The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterization gradient fit Gaussian distributions. In this paper, we introduce the generalized reparameterization gradient, a method that extends the reparameterization gradient to a wider class of variational distributions. Generalized reparameterizations use invertible transformations of the latent variables which lead to transformed distributions that weakly depend on the variational parameters. This results in new Monte Carlo gradients that combine reparameterization gradients and score function gradients. We demonstrate our approach on variational inference for two complex probabilistic models. The generalized reparameterization is effective: even a single sample from the variational distribution is enough to obtain a low-variance gradient.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Predictive Sampling with Forecasting Autoregressive Models (2020, background): “Although we focus on the discrete setting here, reparametrization noise can be obtained for many common continuous probability distributions (Ruiz et al., 2016).”
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (2016, methodology): “Recently, new hybrid estimators have also been developed for continuous latent variables which are not directly reparameterizable, by combining partial reparameterizations with score function estimators [36, 30].”
SHOPPER: A Probabilistic Model of Consumer Choice with Substitutes and Complements (2017, methodology): “Here we have considered the generalized reparameterization approach (Ruiz, Titsias and Blei, 2016; Naesseth et al., 2017), which allows the auxiliary distribution π ( (cid:15) ; ν ) to depend on the variational parameters ν (this is necessary because the gamma random variables are not otherwise…”
PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos (2019, neutral): “…– LR gradients perform better than RP with back-propagation – are contradictory to recent work in stochastic variational inference, which suggest that even a single sample point yields a good gradient estimate using the RP trick (Kingma & Welling, 2014; Rezende et al., 2014; Ruiz et al., 2016).”
Stochastic Gradient Descent as Approximate Bayesian Inference (2017, methodology): “A modiﬁed form of black-box variational inference relies on re-parameterization gradients (Salimans and Knowles, 2013; Kingma and Welling, 2014; Rezende et al., 2014; Kucukelbir et al., 2015; Ruiz et al., 2016).”
Backpropagation through the Void: Optimizing control variates for black-box gradient estimation (2017, methodology): “Or, following Ruiz et al. (2016) and Tucker et al. (2017), we can construct an unbiased, single-sample estimator using the fact that our gradient estimator is unbiased.”
Deep Generative Models (2022, unknown): “Generalized Reparameterization Gradient (G-REP) (Ruiz et al., 2016)”
Generative Modeling of Infinite Occluded Objects for Compositional Scene Representation (2019, methodology): “The first is to utilize the generalized reparameterization gradient (Ruiz et al., 2016) to obtain lowvariance estimate of the gradient.”
Advances in Variational Inference (2017, background): “The generalized reparameterization gradient [166] achieves this by finding an invertible transformation between the noise and the latent variable of interest.”
Monte Carlo Gradient Estimation in Machine Learning (2019, methodology): “Other generalisations like that of Ruiz et al. (2016), Naesseth et al. (2017) and Parmas et al. (2018), combine the score function method with the pathwise methods; we come back to these hybrid methods in Section 8.”
End-to-End Differentiable Adversarial Imitation Learning (2017, background): “1A general version of the re-parametrization trick for other distributions such as beta or gamma was recently proposed by Ruiz et al. (2016)”
WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling (2018, background): “Despite being able to infer a multilayer representation of a text corpus with scalable inference (Patterson & Teh, 2013; Ruiz et al., 2016; Cong et al., 2017a), they usually rely on an iterative procedure to infer the latent representation of a new document at the testing stage, regardless of whethe”
Gradient Estimation with Stochastic Softmax Tricks (2020, methodology): “The Gumbel-Softmax estimator is the simplest; it continuously approximates the GumbelMax trick to admit a reparameterization gradient [37, 68, 72].”
Variance reduction properties of the reparameterization trick (2018, background): “There is now a large literature applying the RP trick successfully in diﬀerent settings and recently it has been extended to a wider range of variational approximations (Ruiz et al., 2016; Figurnov et al., 2018) and even for non-diﬀerentiable models (Lee et al., 2018).”
Variational Sequential Monte Carlo (2017, methodology): “With this assumption, rewrite the gradient of (6) by using the reparameterization trick [Kingma and Welling, 2014, Rezende et al., 2014], This expansion follows from the product rule, just as in the generalized reparameterizations of Ruiz et al. [2016] and Naesseth et al. [2017].”
Wild Variational Approximations (2016, methodology): “For non-reparameterizable distributions ∇φf can be computed with further approximations such as the generalised reparameterization trick [30].”
Variational Rejection Sampling (2018, unknown): “…al., 2017] and deriving gradient estimators that interpolate between score function estimators and reparameterization estimators for distributions that can be simulated using acceptance-rejection algorithms, such as the Gamma and Dirichlet distributions [Ruiz et al., 2016, Naesseth et al., 2017].”
Adversarial Message Passing For Graphical Models (2016, background): “[27] Francisco JR Ruiz, Michalis K Titsias, and David M Blei.”