Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model

By Erik Nijkamp, Mitch Hill, Song-Chun Zhu, Ying Nian Wu. 2019

236 citations recorded by the source corpus.

This paper studies a curious phenomenon in learning energy-based model (EBM) using MCMC. In each learning iteration, we generate synthesized examples by running a non-convergent, non-mixing, and non-persistent short-run MCMC toward the current model, always starting from the same initial distribution such as uniform noise distribution, and always running a fixed number of MCMC steps. After generating synthesized examples, we then update the model parameters according to the maximum likelihood learning gradient, as if the synthesized examples are fair samples from the current model. We treat this non-convergent short-run MCMC as a learned generator model or a flow model. We provide arguments for treating the learned non-convergent short-run MCMC as a valid model. We show that the learned short-run MCMC is capable of generating realistic images. More interestingly, unlike traditional EBM or MCMC, the learned short-run MCMC is capable of reconstructing observed images and interpolating between images, like generator or flow models. The code can be found in the Appendix.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Denoising Diffusion Probabilistic Models (2020, methodology): “Other methods for learning transition operators of Markov chains include infusion training [2], variational walkback [13], generative stochastic networks [1], and others [47, 51, 33, 39].”
Residual Energy-Based Models for Text Generation (2020, methodology): “, 2003) and Langevyn dynamics (Ranzato et al., 2007; Du & Mordatch, 2019; Xie et al., 2016; 2017; 2019; 2018; Gao et al., 2018; Nijkamp et al., 2019) are not applicable when the input is discrete like in text applications.”
Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, methodology): “Samples are typically generated from p ( x ) by an MCMC algorithm.”
Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One (2019, methodology): “Following Du & Mordatch (2019) we use persistent contrastive divergence (Tieleman, 2008) to estimate the expectation in the right-hand-side of Equation 2 since it gives an order of magnitude savings in computation compared to seeding new chains at each iteration as in Nijkamp et al. (2019b).”
Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, methodology): “[56] investigated a similar model to our models based on HIS, although the training algorithm was different.”
On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models (2019, methodology): “Our recent companion work [24] thoroughly explores the capabilities of non-convergent ML.”
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “Recently, it was shown that EBMs with deep neural networks as energy function can successfully model complex data such as natural images (Du & Mordatch, 2019; Nijkamp et al., 2019b;a).”
Hybrid Discriminative-Generative Training via Contrastive Learning (2020, background): “While there are advances in scaling the training of EBMs to to high-dimensional data [34, 6, 8], it remains computational expensive and often costs orders magnitude of more time and computation to training EBMs and therefore makes it difﬁcult to scale.”
Is Conditional Generative Modeling all you need for Decision-Making? (2022, methodology): “The denoising procedure can be alternatively interpreted as parameterizing the gradients of the data distribution (Song et al., 2021) optimizing the score matching objective (Hyv¨arinen, 2005) and thus as a Energy-Based Model (Du & Mordatch, 2019; Nijkamp et al., 2019; Grathwohl et al., 2020).”
3D Shape Generation and Completion through Point-Voxel Diffusion (2021, background): “Energy based models [20, 9, 28] learn an energy landscape over input data, where local minima correspond to high-fidelity samples, which are obtained by Langevin dynamics [9, 28].”
Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models (2020, methodology): “The EBM (1) used in this work is introduced in [28] and important observations about the learning process are presented in [20, 21].”
How to Train Your Energy-Based Model for Regression (2020, unknown): “ML-IS ML-MCMC-1 Table S3: Ablation study for ML-MCMC-1, on the 2017 val split of COCO [13].”
Energy-Based Open-World Uncertainty Modeling for Confidence Calibration (2021, methodology): “Following prior work [33, 6, 12], we also leverage SGLD to optimize our energy-based objective function.”
Planning with Diffusion for Flexible Behavior Synthesis (2022, methodology): “The denoising procedure can be seen as parameterizing the gradients of the data distribution (Song & Ermon, 2019), connecting diffusion models to score matching (Hyv¨arinen, 2005) and EBMs (LeCun et al., 2006; Du & Mordatch, 2019; Nijkamp et al., 2019; Grathwohl et al., 2020).”
How to Train Your Energy-Based Models (2021, methodology): “EBMs trained with CD may not capture the data distribution faithfully, since truncated MCMC can lead to biased gradient updates that hurt the learning dynamics (Schulz et al., 2010; Fischer and Igel, 2010; Nijkamp et al., 2019).”
Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energy (2020, methodology): “In particular, we follow [16, 17] and use short run non convergent MCMC to train our EBMs.”
Unpaired Image-to-Image Translation via Latent Energy Transport (2020, background): “Energy-based Model Recently, energy-based generative models have drawn significant attention [40, 39, 9, 29, 7, 11, 43, 41].”
Energy-Based Imitation Learning (2020, background): “More specifically, MaxEnt IRL models the paths in demonstrations using a Boltzmann distribution, where the energy is given by the unknown reward function r̂∗: pr̂(τ) = 1 Z exp (r̂∗(τ)) , (14) where τ = {s0, a0, · · · , sT , aT } is a trajectory of state-action pairs and T is its length; r̂∗(τ) =∑ t ”
Residual Energy-Based Models for Text (2021, methodology): “, 2003) and Langevyn dynamics (Ranzato et al., 2007; Du and Mordatch, 2019; Xie et al., 2016, 2017, 2019, 2018; Gao et al., 2018; Nijkamp et al., 2019) are not applicable when the input is discrete like in text applications.”