Denoising Diffusion Probabilistic Models
By Jonathan Ho, Ajay Jain, Pieter Abbeel. Neural Information Processing Systems, 2020
33,172 citations recorded by the source corpus.
We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics. Our best results are obtained by training on a weighted variational bound designed according to a novel connection between diffusion probabilistic models and denoising score matching with Langevin dynamics, and our models naturally admit a progressive lossy decompression scheme that can be interpreted as a generalization of autoregressive decoding. On the unconditional CIFAR10 dataset, we obtain an Inception score of 9.46 and a state-of-the-art FID score of 3.17. On 256x256 LSUN, we obtain sample quality similar to ProgressiveGAN. Our implementation is available at this https URL
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models, Probabilistic & Bayesian Methods
View Paper: https://arxiv.org/abs/2006.11239
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Sergey Levine
- Mohit Bansal
- Jianfeng Gao
- Quoc V. Le
- Percy Liang
- Ruslan Salakhutdinov
- Dacheng Tao
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
- Andrea Vedaldi
In Statistical and Nonlinear Physics
- Arash Vahdat
- Yang Song
- Stefano Ermon
- Jonathan Ho
- Karsten Kreis
- Diederik P. Kingma
- Prafulla Dhariwal
- Jun Zhu
In Generative Models
- Arash Vahdat
- Stefano Ermon
- Jun-Yan Zhu
- Karsten Kreis
- Richard Zhang
- Eli Shechtman
- Jiaming Song
- Jan Kautz
In Probabilistic & Bayesian Methods
- Stefano Ermon
- Yoshua Bengio
- Yang Song
- Arash Vahdat
- Diederik P. Kingma
- Aaron C. Courville
- Max Welling
- Jiaming Song
Citations in the corpus
- High-Resolution Image Synthesis with Latent Diffusion Models (2021, methodology): “Following [15], the UNet of this diffusion model uses attention layers on three levels of its feature hierarchy, the BigGAN [3] residual block for up- and downsampling and has 387M parameters instead of 215M.”
- Classifier-Free Diffusion Guidance (2022, background): “The reverse process mean comes from an estimate xθ(zλ) ≈ x plugged into q(zλ′ |zλ,x) [5, 8] (xθ also receives λ as input, but we suppress this to keep our notation clean).”
- Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors (2022, methodology): “Other likelihood-based models include autoregressive models [58, 43, 13, 8] and diffusion models [11, 21, 20].”
- GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models (2021, methodology): “(2015) and improved by Song & Ermon (2020b); Ho et al. (2020). Given a sample from the data distribution x0 ∼ q(x0), we produce a Markov chain of latent variables x1, .”
- Variational Diffusion Models (2021, methodology): “This diffusion process specification includes the variance-preserving diffusion process as used by [Sohl-Dickstein et al., 2015, Ho et al., 2020] as a special case, where ↵t = p 1 2t .”
- D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, methodology): “Our D2C implementation is based on the state-of-the-art NVAE [91] autoencoder structure, the U-Net diffusion model [42], and the MoCo-v2 contrastive representation learning method [16].”
- Score-based Generative Modeling in Latent Space (2021, methodology): “Similar to [1, 2], we observe that when w ( t ) is dropped while training the SGM prior (i.e., w ( t ) = 1 ), LSGM often yields higher quality samples at a small cost in likelihood.”
- On Fast Sampling of Diffusion Probabilistic Models (2021, methodology): “Diffusion probabilistic models are a class of deep generative models that use Markov chains to gradually transform between a simple distribution (e.g., isotropic Gaussian) and the complex data distribution (Sohl-Dickstein et al., 2015; Ho et al., 2020).”
- Cascaded Diffusion Models for High Fidelity Image Generation (2021, methodology): “Scalar conditioning, such as a class label or a diffusion timestep t, is provided by adding embeddings into intermediate layers of the network [10].”
- High-Resolution Complex Scene Synthesis with Transformers (2021, methodology): “Architectures The architectural design of our models follows [7]: For the VQGAN , we use the encoder-decoder structure as introduced in [10] (without skip connections) and a patch-discriminator as in [33].”
- Diffusion Models Beat GANs on Image Synthesis (2021, methodology): “[31] observe that the simple mean-squared error objective, Lsimple, works better in practice than the actual variational lower bound Lvlb that can be derived from interpreting the denoising diffusion model as a VAE.”
- VideoGPT: Video Generation using VQ-VAE and Transformers (2021, neutral): “, 2016) have seen incredible progress in the last few years on multiple modalities including natural images (van den Oord et al., 2016c; Zhang et al., 2019; Brock et al., 2018; Kingma & Dhariwal, 2018; Ho et al., 2019a; Karras et al., 2017; 2019; Van Den Oord et al., 2017; Razavi et al., 2019; Vahda”
- Image Super-Resolution via Iterative Refinement (2021, methodology): “The SR3 architecture is similar to the U-Net in DDPM [1], with self-attention and modifications adapted from [40]; i.e., we replace the original DDPM residual blocks with residual blocks from BigGAN [41], and we re-scale skip connections by 1ffiffi 2 p .”
- Noise Estimation for Generative Diffusion Models (2021, methodology): “For image generation, we use denoising diffusion models trained in [13], relying on the implementation available in [12].”
- Symbolic Music Generation with Diffusion Models (2021, methodology): “DDPMs (Sohl-Dickstein et al., 2015; Ho et al., 2020) are a class of generative models that define latents x1, .”
- Score-Based Generative Modeling through Stochastic Differential Equations (2020, positive): “The model tested here is a DDPM model trained with the same settings in Ho et al. (2020).”
- Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images (2020, positive): “Ho et al. (2020) showed that such models achieve impressive sample quality with great depth, which is in line with our observations that greater depth is helpful for VAEs.”