On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models

By Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, Ying Nian Wu. 2019

185 citations recorded by the source corpus.

This study investigates the effects of Markov chain Monte Carlo (MCMC) sampling in unsupervised Maximum Likelihood (ML) learning. Our attention is restricted to the family of unnormalized probability densities for which the negative log density (or energy function) is a ConvNet. We find that many of the techniques used to stabilize training in previous studies are not necessary. ML learning with a ConvNet potential requires only a few hyper-parameters and no regularization. Using this minimal framework, we identify a variety of ML learning outcomes that depend solely on the implementation of MCMC sampling. On one hand, we show that it is easy to train an energy-based model which can sample realistic images with short-run Langevin. ML can be effective and stable even when MCMC samples have much higher energy than true steady-state samples throughout training. Based on this insight, we introduce an ML method with purely noise-initialized MCMC, high-quality short-run synthesis, and the same budget as ML with informative MCMC initialization such as CD or PCD. Unlike previous models, our energy model can obtain realistic high-diversity samples from a noise signal after training. On the other hand, ConvNet potentials learned with non-convergent MCMC do not have a valid steady-state and cannot be considered approximate unnormalized densities of the training data because long-run MCMC samples differ greatly from observed images. We show that it is much harder to train a ConvNet potential to learn a steady-state over realistic images. To our knowledge, long-run MCMC samples of all previous models lose the realism of short-run samples. With correct tuning of Langevin noise, we train the first ConvNet potentials for which long-run and steady-state MCMC samples are realistic images.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Denoising Diffusion Probabilistic Models (2020, neutral): “By the known connection between score matching and energy-based modeling, our work could have implications for other recent work on energy-based models [11, 38, 15, 8].”
Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One (2019, methodology): “Following Du & Mordatch (2019) we use persistent contrastive divergence (Tieleman, 2008) to estimate the expectation in the right-hand-side of Equation 2 since it gives an order of magnitude savings in computation compared to seeding new chains at each iteration as in Nijkamp et al. (2019b).”
Generative Modeling by Estimating Gradients of the Data Distribution (2019, methodology): “(4), but it can often be ignored in practice [9, 12, 39].”
Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model (2019, background): “[35] performs a thorough investigation of various persistent and non-persistent, as well as convergent and non-convergent learning schemes.”
On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, background): “A wide range of image and video synthesis applications [45, 54, 67, 83] have been enabled, as a result of tremendous progress in deep generative models such as GANs [9, 28, 34, 37, 59], VAEs [41, 55, 61], autoregressive models [52], flow-based models [18, 40], and energy-based models [21, 50, 63].”
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “Recently, it was shown that EBMs with deep neural networks as energy function can successfully model complex data such as natural images (Du & Mordatch, 2019; Nijkamp et al., 2019b;a).”
Learning Energy-Based Models by Diffusion Recovery Likelihood (2020, methodology): “CoopNets (Xie et al., 2019) - 7.30 EBM-IG (Du & Mordatch, 2019) 37.9 8.30 JEM (Grathwohl et al., 2019) 38 Setting / Objective FID ↓ Inception ↑ T = 1, K = 180 32.12 6.89 ± 0.08 T = 1000, K = 0 22.58 7.86 ± 0.11 T = 1000, K = 0 (DSM) 21.76 7.80 ± 0.07 T = 6, K = 10 - -”
Learning Gradient Fields for Shape Generation (2020, neutral): “For inference, we set T = 10 and α = 2 × 10 − 4 .”
COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics (2022, unknown): “Common gradient-free Markov chain Monte Carlo (MCMC) methods such as Gibbs sampling (Bishop & Nasrabadi, 2006) can be used, but they are often prohibitively slow (Duvenaud et al., 2021; Nijkamp et al., 2020).”
Adversarial Fisher Vectors for Unsupervised Representation Learning (2019, background): “There has also been an increasing in the interest of deep EBMs trained with traditional sampling approaches, see [20, 21].”
Further Analysis of Outlier Detection with Deep Generative Models (2020, unknown): “While we conjecture that VAEs or deep AR models can exhibit similar failures due to a different type of inductive bias, we cannot reverse the above experiment and train these models on EBM samples, as sampling from EBMs rely on ad hoc processes such as premature termination of MCMC chains [5, 6, 26]”
Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models (2020, unknown): “Furthermore, when the model is fully visible, scalable methods (Du et al., 2018; Du & Mordatch, 2019; Nijkamp et al., 2020; 2019; Grathwohl et al., 2020a) are applicable by estimating such gradient with Langevin dynamics.”
Energy-Based Models for Deep Probabilistic Regression (2020, methodology): “Recently, energy-based models have been used primarily for unsupervised learning problems within computer vision [58, 14, 11, 29, 40], where DNNs are directly used to predict Eθ(x).”
On Learning Non-Convergent Short-Run MCMC Toward Energy-Based Model (2019, background): “See the recent work [36] on a thorough diagnosis of various persistent and non-persistent, as well as convergent and non-convergent implementations of MCMC for learning EBM.”
Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energy (2020, methodology): “Recently, [7, 16, 17] successfully scales up the maximum likelihood learning of EBMs to high dimensional images.”
Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models (2020, methodology): “Our work builds on the convergent learning methodology in [20] to apply long-run Langevin sampling as a defense technique.”
Compositional Visual Generation with Composable Diffusion Models (2022, methodology): “Energy-Based Models (EBMs) [10,9,12,28] are a class of generative models where the data distribution is modeled using an unnormalized probability density.”
Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling (2020, methodology): “This changes the update rule to: This temperature-scaling approach has been used in most recent work on large-scale EBMs (Nijkamp et al., 2019a; Grathwohl et al., 2019; Nijkamp et al., 2019b; Du & Mor-datch, 2019).”