An introduction to deep generative modeling

By Lars Ruthotto, Eldad Haber. GAMM-Mitteilungen, 2021

305 citations recorded by the source corpus.

Deep generative models (DGM) are neural networks with many hidden layers trained to approximate complicated, high‐dimensional probability distributions using samples. When trained successfully, we can use the DGM to estimate the likelihood of each observation and to create new samples from the underlying distribution. Developing DGMs has become one of the most hotly researched fields in artificial intelligence in recent years. The literature on DGMs has become vast and is growing rapidly. Some advances have even reached the public sphere, for example, the recent successes in generating realistic‐looking images, voices, or movies; so‐called deep fakes. Despite these successes, several mathematical and practical issues limit the broader use of DGMs: given a specific dataset, it remains challenging to design and train a DGM and even more challenging to find out why a particular model is or is not effective. To help advance the theoretical understanding of DGMs, we introduce DGMs and provide a concise mathematical framework for modeling the three most popular approaches: normalizing flows, variational autoencoders, and generative adversarial networks. We illustrate the advantages and disadvantages of these basic approaches using numerical experiments. Our goal is to enable and motivate the reader to contribute to this proliferating research area. Our presentation also emphasizes relations between generative modeling and optimal transport.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/988a1ab0c1ebd7eb4817fbc8ce41f962dd519998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Gero Strobel
2. Dongchao Yang
3. Qichen Ye
4. Bowen Cao
5. Leonardo Banh
6. Yuexian Zou
7. Wen Wang

In Computer Vision and Pattern Recognition

1. Gero Strobel
2. Dongchao Yang
3. Qichen Ye
4. Bowen Cao
5. Leonardo Banh
6. Yuexian Zou
7. Wen Wang

In Statistical and Nonlinear Physics

1. Gero Strobel
2. Dongchao Yang
3. Leonardo Banh
4. Yuexian Zou
5. Wen Wang

Citations in the corpus

• NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement (2023, background): “…approaches [9]–[11] employ different infrastructures such as Generative Adversarial Networks (GANs) [12], Variational Autoencoders (VAEs) [13] and flow-based models [14] to learn the distribution of clean speech signals, which can be more robust to complex and varying noise scenarios [15].”
• JFB: Jacobian-Free Backpropagation for Implicit Networks (2021, unknown): “Other Implicit Formulations A related implicit learning formulation is the well-known neural ODE model (Chen et al. 2018; Dupont, Doucet, and Teh 2019; Ruthotto and Haber 2021).”
• Generative artificial intelligence (2023, methodology): “By leveraging statistics, the goal of DGM training is to learn high-dimensional probability distributions from a finite training dataset and create new, similar samples that resemble an approximation to the underlying class of training data (Ruthotto & Haber, 2021).”