On Learning Non-Convergent Short-Run MCMC Toward Energy-Based Model

By Erik Nijkamp, Song-Chun Zhu, Ying Nian Wu. 2019

29 citations recorded by the source corpus.

Subfields: Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One (2019, methodology): “Following Du & Mordatch (2019) we use persistent contrastive divergence (Tieleman, 2008) to estimate the expectation in the right-hand-side of Equation 2 since it gives an order of magnitude savings in computation compared to seeding new chains at each iteration as in Nijkamp et al. (2019b).”
Flow Contrastive Estimation of Energy-Based Models (2019, background): “[55, 9] scale up sampling-based methods to large image datasets with white noise as the starting point of sampling.”
Learning Energy-Based Models by Diffusion Recovery Likelihood (2020, methodology): “CoopNets (Xie et al., 2019) - 7.30 EBM-IG (Du & Mordatch, 2019) 37.9 8.30 JEM (Grathwohl et al., 2019) 38 Setting / Objective FID ↓ Inception ↑ T = 1, K = 180 32.12 6.89 ± 0.08 T = 1000, K = 0 22.58 7.86 ± 0.11 T = 1000, K = 0 (DSM) 21.76 7.80 ± 0.07 T = 6, K = 10 - -”
Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC (2020, methodology): “Taking advantage of the approximation capacity of modern deep networks such as convolutional network (ConvNet) [39, 34], recent papers [57, 13, 36, 47, 11] parametrize the energy function by ConvNet.”
Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling (2020, methodology): “This changes the update rule to: This temperature-scaling approach has been used in most recent work on large-scale EBMs (Nijkamp et al., 2019a; Grathwohl et al., 2019; Nijkamp et al., 2019b; Du & Mor-datch, 2019).”
Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models (2020, unknown): “Furthermore, when the model is fully visible, scalable methods (Du et al., 2018; Du & Mordatch, 2019; Nijkamp et al., 2020; 2019; Grathwohl et al., 2020a) are applicable by estimating such gradient with Langevin dynamics.”
Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, background): “Energy-based Models (EBMs) [26, 48, 58, 64, 67, 68, 75, 78, 79, 120, 129, 132, 165, 170, 182, 196, 254, 281] can be viewed as one generative version of discriminators [79, 104, 131, 134], while can be learned from unlabeled input data.”