Reparameterization Gradient for Non-differentiable Models
By Wonyeol Lee, Hangyeol Yu, Hongseok Yang. 2018
38 citations recorded by the source corpus.
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the reparameterization trick, one of the most effective techniques for addressing the variance issue for differentiable models, so that the trick works for non-differentiable models as well. Our algorithm splits the space of latent variables into regions where the density of the variables is differentiable, and their boundaries where the density may fail to be differentiable. For each differentiable region, the algorithm applies the standard reparameterization trick and estimates the gradient restricted to the region. For each potentially non-differentiable boundary, it uses a form of manifold sampling and computes the direction for variational parameters that, if followed, would increase the boundary's contribution to the variational objective. The sum of all the estimates becomes the gradient estimate of our algorithm. Our estimator enjoys the reduced variance of the reparameterization gradient while remaining unbiased even for non-differentiable models. The experiments with our preliminary implementation confirm the benefit of reduced variance and unbiasedness.
Subfields: Artificial Intelligence, Statistical and Nonlinear Physics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Chris J. Maddison
- Daniel Tarlow
- Max Welling
- Andriy Mnih
- Ruslan Salakhutdinov
- Sergey Levine
- Miltiadis Allamanis
In Statistical and Nonlinear Physics
- Andriy Mnih
- Chris J. Maddison
- Shakir Mohamed
- Mihaela Rosca
- Daniel Tarlow
- Scott Anthony Sisson
- Andreas Krause
- Michael Figurnov
Citations in the corpus
- Gradient Estimation with Stochastic Softmax Tricks (2020, background): “Instead it is the jump discontinuities in the argmax that invalidate this particular exchange of expectation and differentiation [47, 8, Chap.”
- Monte Carlo Gradient Estimation in Machine Learning (2019, background): “In the variational inference setting, Lee et al. (2018) also look an the non-differentiable case by splitting regions in to differentiable and non-differentiable components.”
- Variance reduction properties of the reparameterization trick (2018, background): “, 2018) and even for non-differentiable models (Lee et al., 2018).”