Understanding Diffusion Models: A Unified Perspective

By Calvin Luo. arXiv.org, 2022

522 citations recorded by the source corpus.

Diffusion models have shown incredible capabilities as generative models; indeed, they power the current state-of-the-art models on text-conditioned image generation such as Imagen and DALL-E 2. In this work we review, demystify, and unify the understanding of diffusion models across both variational and score-based perspectives. We first derive Variational Diffusion Models (VDM) as a special case of a Markovian Hierarchical Variational Autoencoder, where three key assumptions enable tractable computation and scalable optimization of the ELBO. We then prove that optimizing a VDM boils down to learning a neural network to predict one of three potential objectives: the original source input from any arbitrary noisification of it, the original source noise from any arbitrarily noisified input, or the score function of a noisified input at any arbitrary noise level. We then dive deeper into what it means to learn the score function, and connect the variational perspective of a diffusion model explicitly with the Score-based Generative Modeling perspective through Tweedie's Formula. Lastly, we cover how to learn a conditional distribution using diffusion models via guidance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, General Social Sciences

View Paper: https://www.semanticscholar.org/paper/17d068e78e6f25e65cb08319b19b58279bb8b214

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Tat-seng Chua
3. Fuli Feng
4. Xiang Wang
5. Zhen Dong
6. Kurt Keutzer
7. Bin Cui
8. Wenjie Wang

In Computer Vision and Pattern Recognition

1. Tat-seng Chua
2. Xiangnan He
3. Kurt Keutzer
4. Ming-Hsuan Yang
5. Fuli Feng
6. Luc Van Gool
7. Dacheng Tao
8. Cordelia Schmid

In General Social Sciences

1. Tat-seng Chua
2. Kurt Keutzer
3. Wenjie Wang
4. Fuli Feng
5. Yiyan Xu
6. Xinyu Lin
7. Xiangnan He

Citations in the corpus

• Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, background): “For a detailed derivation, one can refer to [5, 138].”
• Q-Diffusion: Quantizing Diffusion Models (2023, background): “We refer readers to [29] for a more detailed introduction.”
• Diffusion Recommender Model (2023, methodology): “Thanks to the reparameterization trick [11] and the additivity of two independent Gaussian noises [11, 25], we can directly obtain xt from x0.”