Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net

By Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio. 2017

57 citations recorded by the source corpus.

We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model whose stationary distribution obeys detailed balance with respect to a parameterized energy function. The energy function is then modified so the model and data distributions match, with no guarantee on the number of steps required for the Markov chain to converge. Moreover, the detailed balance condition is highly restrictive: energy based models corresponding to neural networks must have symmetric weights, unlike biological neural circuits. In contrast, we develop a method for directly learning arbitrarily parameterized transition operators capable of expressing non-equilibrium stationary distributions that violate detailed balance, thereby enabling us to learn more biologically plausible asymmetric neural networks and more general non-energy based dynamical systems. The proposed training objective, which we derive via principled variational methods, encourages the transition operator to "walk back" in multi-step trajectories that start at data-points, as quickly as possible back to the original data points. We present a series of experimental results illustrating the soundness of the proposed approach, Variational Walkback (VW), on the MNIST, CIFAR-10, SVHN and CelebA datasets, demonstrating superior samples compared to earlier attempts to learn a transition operator. We also show that although each rapid training trajectory is limited to a finite but variable number of steps, our transition operator continues to generate good samples well past the length of such trajectories, thereby demonstrating the match of its non-equilibrium stationary distribution to the data distribution. Source Code: this http URL

Subfields: Artificial Intelligence, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Denoising Diffusion Probabilistic Models (2020, methodology): “Other methods for learning transition operators of Markov chains include infusion training [2], variational walkback [13], generative stochastic networks [1], and others [47, 51, 33, 39].”
Flow Contrastive Estimation of Energy-Based Models (2019, background): “On the other end of the spectrum of deep generative models lies the family of energy-based models (EBMs) [42, 54, 29, 77, 73, 13, 38, 55, 9, 11, 16, 18, 6].”
Generative Modeling by Estimating Gradients of the Data Distribution (2019, methodology): “Previous approaches such as Infusion Training (IT [5]) and Variational Walkback (VW [16]) also employed different noise levels/temperatures for training transition operators of a Markov chain.”
On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models (2019, methodology): “tentials. Earlier energy-based models such as RBM do not exhibit a dramatic difference in realism between shortrun samples from informative initialization and steadystate images. Variational Walkback [11] is an energy-free method that can directly learn a MCMC sampling process with a realistic stead”
On Fast Sampling of Diffusion Probabilistic Models (2021, background): “Diffusion models are a class of powerful deep generative models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Goyal et al., 2017), which have received a lot of attention recently.”
Diffusion Models Beat GANs on Image Synthesis (2021, methodology): “[27] described a technique for learning a model with learned iterative generation steps, and found that it could achieve good image samples when trained with a likelihood objective.”
Score-Based Generative Modeling through Stochastic Differential Equations (2020, neutral): “Score-based generative models, and related techniques (Bordes et al., 2017; Goyal et al., 2017; Du & Mordatch, 2019), have proven effective at generation of images (Song & Ermon, 2019; 2020; Ho et al., 2020), audio (Chen et al., 2020; Kong et al., 2020), graphs (Niu et al., 2020), and shapes (Cai ˚…”
Denoising Diffusion Implicit Models (2020, methodology): “Our work is based on a large family of existing methods on learning generative models as transition operators of Markov chains (Sohl-Dickstein et al., 2015; Bengio et al., 2014; Salimans et al., 2014; Song et al., 2017; Goyal et al., 2017; Levy et al., 2017).”
DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, background): “, isotropic Gaussian) into complicated data distribution (Sohl-Dickstein et al., 2015; Goyal et al., 2017; Ho et al., 2020).”
Improved Denoising Diffusion Probabilistic Models (2021, background): “More recently, Ho et al. (2020) showed an equivalence between denoising diffusion probabilistic models (DDPM) and score based generative models (Song & Ermon, 2019; 2020), which learn a gradient of the logdensity of the data distribution using denoising score matching (Hyvärinen, 2005).”
Adversarial score matching and improved sampling for image generation (2020, methodology): “Li et al. (2019) and Song and Ermon (2020) found that the unconditional network produced better samples, while Ho et al.”
Learning Energy-Based Models by Diffusion Recovery Likelihood (2020, background): “EBMs [27, 33, 21, 47, 10, 43, 6, 7, 25, 35, 5, 11, 4, 8, 2, 12] are an appealing class of probabilistic models, which can be viewed as generative versions of discriminators [18, 26, 28, 12], yet can be learned from unlabeled data.”
A Variational Perspective on Diffusion-Based Generative Models and Score Matching (2021, background): “discrete-time diffusion-based generative models (Sohl-Dickstein et al., 2015; Goyal et al., 2017; Ho et al., 2020) can be viewed as learning to revert an inference process (using the plug-in reverse SDE).”
Structured Denoising Diffusion Models in Discrete State-Spaces (2021, background): “[15] proposed a diffusion-like model for images with a more flexible family of learned corruption processes.”
On Learning Non-Convergent Short-Run MCMC Toward Energy-Based Model (2019, methodology): “The variational walkback method [14] is an energy-free method that can directly learn a MCMC sampling process.”
A Survey on Generative Diffusion Models (2022, background): “Within this survey, our central emphasis lies on diffusion models, which epitomize the forefront of advancements within this domain.”
Score-Based Generative Modeling with Critically-Damped Langevin Diffusion (2021, background): “Generative modeling by learning stochastic processes has a long history (Movellan, 2008; Lyu, 2009; Sohl-Dickstein et al., 2011; Bengio et al., 2014; Alain et al., 2016; Goyal et al., 2017; Bordes et al., 2017; Song & Ermon, 2019; Ho et al., 2020).”
Recall Traces: Backtracking Models for Efficient Reinforcement Learning (2018, methodology): “(Goyal et al., 2017) proposed to use similar learning rule in the context of generative models to modify the parameters of transition operator to make the reverse of this heated trajectory more likely under a reverse cooling process.”
Variational Autoencoder with Arbitrary Conditioning (2018, background): “Rezende et al. (2014) [Appendix F], Sohl-Dickstein et al. (2015), Goyal et al. (2017), and Bordes et al. (2017) propose to ﬁll missing data with noise and run Markov chain with a learned transition operator.”