Generalized Energy Based Models

By Michael Arbel, Liang Zhou, Arthur Gretton. 2020

97 citations recorded by the source corpus.

We introduce the Generalized Energy Based Model (GEBM) for generative modelling. These models combine two trained components: a base distribution (generally an implicit model), which can learn the support of data with low intrinsic dimension in a high dimensional space; and an energy function, to refine the probability mass on the learned support. Both the energy function and base jointly constitute the final model, unlike GANs, which retain only the base distribution (the "generator"). GEBMs are trained by alternating between learning the energy and the base. We show that both training stages are well-defined: the energy is learned by maximising a generalized likelihood, and the resulting energy-based loss provides informative gradients for learning the base. Samples from the posterior on the latent space of the trained model can be obtained via MCMC, thus finding regions in this space that produce better quality samples. Empirically, the GEBM samples on image-generation tasks are of much better quality than those from the learned generator alone, indicating that all else being equal, the GEBM will outperform a GAN of the same complexity. GEBMs also return state-of-the-art performance on density modelling tasks, and when using base measures with an explicit form.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “Zhao et al. (2017); Che et al. (2020); Song et al. (2020); Arbel et al. (2020) formulate EBMs with GANs, and use the discriminator to assign an energy.”
Energy-Based Open-World Uncertainty Modeling for Confidence Calibration (2021, methodology): “Energy-Based Models (EBMs) have been widely used in the area of generative modeling [41, 8, 1, 4].”
Controllable and Compositional Generation with Latent-Space Energy-Based Models (2021, background): “Also, many other works have built EBMs or score-based models in the latent space of generative models, most of which, however, focus on improving generation quality with EBMs as a structural prior [2, 4, 19, 56, 55, 58, 53].”
Improved Contrastive Divergence Training of Energy Based Models (2020, background): “…issue, a separate line of work utilizes an auxiliary network to amortize the negative portions of the sampling procedure (Kim & Bengio, 2016; Kumar et al., 2019; Han et al., 2019; Xie et al., 2018a; Song & Ou, 2018; Dai et al., 2019; Che et al., 2020; Grathwohl et al., 2020a; Arbel et al., 2020).”
Pre-training Molecular Graph Representation with 3D Geometry (2021, unknown): “18 [2] Michael Arbel, Liang Zhou, and Arthur Gretton.”
Refining Deep Generative Models via Discriminator Gradient Flow (2020, methodology): “In energy-based generative models (Arbel et al., 2021; Deng et al., 2020), the energy functions can be viewed as a component that improves some base generator.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “This limits the freedom of the model, however, leading some to jointly sample in latent and data space [3], [245].”