Reweighted Wake-Sleep

By Jörg Bornschein, Yoshua Bengio. 2014

198 citations recorded by the source corpus.

Training deep directed graphical models with many hidden variables and performing inference remains a major challenge. Helmholtz machines and deep belief networks are such models, and the wake-sleep algorithm has been proposed to train them. The wake-sleep algorithm relies on training not just the directed generative model but also a conditional generative model (the inference network) that runs backward from visible to latent, estimating the posterior distribution of latent given visible. We propose a novel interpretation of the wake-sleep algorithm which suggests that better estimators of the gradient can be obtained by sampling latent variables multiple times from the inference network. This view is based on importance sampling as an estimator of the likelihood, with the approximate inference network as a proposal distribution. This interpretation is confirmed experimentally, showing that better likelihood can be achieved with this reweighted wake-sleep procedure. Based on this interpretation, we propose that a sigmoidal belief network is not sufficiently powerful for the layers of the inference network in order to recover a good estimator of the posterior distribution of latent variables. Our experiments show that using a more powerful layer model, such as NADE, yields substantially better generative models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

Citations in the corpus

Neural Autoregressive Distribution Estimation (2016, background): “Bornschein and Bengio (2015) showed that NADE could serve as a powerful prior over the latent state of directed graphical model.”
GSNs : Generative Stochastic Networks (2015, background): “More recently, it was followed up by the various variational auto-encoders or VAEs (Gregor et al., 2014; Kingma & Welling, 2014; Mnih & Gregor, 2014; Rezende et al., 2014), and related directed models (Bornschein & Bengio, 2014; Ozair & Bengio, 2014).”
Importance Weighted Autoencoders (2015, methodology): “However, Bornschein & Bengio (2015) interpret this update as following a biased estimate of ∇ θ log p ( x ) , whereas we interpret it as following an unbiased estimate of ∇ θ L k .”
Neural Variational Inference for Text Processing (2015, methodology): “Recent advances in neural variational inference mitigate this problem by reparameterising the continuous random variables (Rezende et al., 2014; Kingma & Welling, 2014), using control variates (Mnih & Gregor, 2014) or approximating the posterior with importance sampling (Bornschein & Bengio, 2015).”
MuProp: Unbiased Backpropagation for Stochastic Neural Networks (2015, methodology): “We do not consider the importance-sampling based approaches (Raiko et al., 2015; Tang & Salakhutdinov, 2013; Bornschein & Bengio, 2015; Burda et al., 2015) as these can be interpreted as optimizing a different objective (Burda et al., 2015) that requires sampling the latent variables multiple times.”
Variational Inference for Monte Carlo Objectives (2016, methodology): “Reweighted Wake Sleep: Though the Reweighted Wake Sleep algorithm (RWS, Bornschein & Bengio, 2015) for training generative models has been derived from the perspective of approximating the log-likelihood gradients us-ing importance sampling, it is closely related to the bound optimization approach…”
Maximum-Likelihood Augmented Discrete Generative Adversarial Networks (2017, background): “Similar discussions can be found in (Bornschein & Bengio, 2015; Norouzi et al., 2016a).”
Discrete Variational Autoencoders (2016, methodology): “Neural adaptive importance sampling [31] and reweighted wake-sleep [32] use sophisticated approximations to the gradient of the log-likelihood that do not admit direct backpropagation.”
Boundary-Seeking Generative Adversarial Networks (2017, methodology): “Our discrete solu-tion is highly analogous to reweighted wake-sleep (RWS, Bornschein & Bengio, 2014), a highly successful approach for training Helmholtz machines with discrete variables (but not in a GAN framework).”
Probabilistic Neural Architecture Search (2019, methodology): “The update rules and procedure derived here are similar to the ones used in the reweighted wake-sleep procedure (Bornschein and Bengio, 2015; Le et al., 2018), where rather than updating variational parameters of an approximate posterior distribution we are updating the parameters of the prior.”
Deep Directed Generative Autoencoders (2014, background): “…(RWS) The DGA can be seen as a special case of the Variational Autoencoder (VAE), with various versions introduced by Kingma and Welling (2014); Gregor et al. (2014); Mnih and Gregor (2014); Rezende et al. (2014), and of the Reweighted Wake-Sleep (RWS) algorithm (Bornschein and Bengio, 2014).”
Variational Memory Addressing in Generative Models (2017, methodology): “Alternative gradient estimators for discrete latent variable models (e.g. NVIL [20], RWS [4] or Gumbel-max relaxation-based approaches [19, 12]) might work here too, but we have not investigated their effectiveness.”
Learning Wake-Sleep Recurrent Attention Models (2015, methodology): “inference networks [8], the reweighted wake-sleep algorithm [9], and control variates [10, 11].”
The Thermodynamic Variational Objective (2019, methodology): “The wake-sleep (WS) [18] and reweighted wake-sleep (RWS) [19] algorithms have traditionally been viewed as using different objectives during the wake and sleep phase.”
AUTO-ENCODING VARIATIONAL BAYES (2020, unknown): “This alternate minimization scheme is similar in spirit to the reweighted wake-sleep algorithm [26, 14], although without the sleep phase, and with a more general class of objectives for updating the proposal.”
Training Deep Generative Models: Variations on a Theme (2015, methodology): “[5] Jörg Bornschein and Yoshua Bengio.”
Iterative Refinement of Approximate Posterior for Training Directed Belief Networks (2015, methodology): “Methods for training variants of the Helmholtz machine (Dayan et al., 1995), such as wake-sleep (Hinton et al., 1995; Bornschein & Bengio, 2014) and neural variational inference and learning (NVIL, Mnih & Gregor, 2014), and methods for training variational autoencoders (VAE, Kingma & Welling, 2013)…”