Denoising Diffusion Implicit Models
By Jiaming Song, Chenlin Meng, Stefano Ermon. 2020
12,819 citations recorded by the source corpus.
Denoising diffusion probabilistic models (DDPMs) have achieved high quality image generation without adversarial training, yet they require simulating a Markov chain for many steps to produce a sample. To accelerate sampling, we present denoising diffusion implicit models (DDIMs), a more efficient class of iterative implicit probabilistic models with the same training procedure as DDPMs. In DDPMs, the generative process is defined as the reverse of a Markovian diffusion process. We construct a class of non-Markovian diffusion processes that lead to the same training objective, but whose reverse process can be much faster to sample from. We empirically demonstrate that DDIMs can produce high quality samples $10 \times$ to $50 \times$ faster in terms of wall-clock time compared to DDPMs, allow us to trade off computation for sample quality, and can perform semantically meaningful image interpolation directly in the latent space.
Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Andrea Vedaldi
- Chen Change Loy
In Statistical and Nonlinear Physics
- Arash Vahdat
- Yang Song
- Stefano Ermon
- Jonathan Ho
- Karsten Kreis
- Jun Zhu
- Prafulla Dhariwal
- Diederik P. Kingma
In Generative Models
- Arash Vahdat
- Jun-Yan Zhu
- Stefano Ermon
- Eli Shechtman
- Richard Zhang
- Karsten Kreis
- Alexei A. Efros
- Daniel Cohen-Or
In Signal Processing
Citations in the corpus
- High-Resolution Image Synthesis with Latent Diffusion Models (2021, methodology): “6, we compare models trained on CelebAHQ [38] and ImageNet in terms sampling speed for different numbers of denoising steps with the DDIM sampler [81] and plot it against FID-scores [28].”
- GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models (2021, background): “Most previous work that uses diffusion models for inpainting has not trained diffusion models explicitly for this task (Sohl-Dickstein et al., 2015; Song et al., 2020b; Meng et al., 2021).”
- Generative Adversarial Networks (2021, neutral): “It is worth mentioning also normalizing flows [51] and score based generative models [67, 66, 48], which become very popular recently and show competitive performance with GANs.”
- Variational Diffusion Models (2021, background): “captures all the different objectives discussed by Song et al. [2021b], see Appendix K.”
- D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, methodology): “On conditional generation with 100 labeled examples, D2C significantly outperforms state-of-the-art VAE [91] and diffusion models [84].”
- Score-based Generative Modeling in Latent Space (2021, methodology): “Variants of SGMs have been applied to images [1, 2, 5, 6], audio [7, 8, 9, 10], graphs [11] and point clouds [12, 13].”
- On Fast Sampling of Diffusion Probabilistic Models (2021, methodology): “Song et al. (2020a) introduce denoising diffusion implicit models (DDIM), which contain non-Markovian processes that lead to an equivalent training objective as DDPM.”
- Cascaded Diffusion Models for High Fidelity Image Generation (2021, background): “There have been a number of improvements and alternatives proposed to the diffusion framework, for example generalization to continuous time [28], deterministic sampling [25], adversarial training [11], and others [6].”
- Diffusion Models Beat GANs on Image Synthesis (2021, methodology): “In Section 2, we give a brief background of diffusion models based on Ho et al. [25] and the improvements from Nichol and Dhariwal [43] and Song et al. [57], and we describe our evaluation setup.”
- Noise Estimation for Generative Diffusion Models (2021, background): “Denoising Diffusion Implicit Models (DDIMs) [30] is an acceleration for the denoising process.”
- High Fidelity Visualization of What Your Self-Supervised Representation Knows About (2021, background): “…noise image, and takes multiple steps in input space that can be thought of as (noisy) gradient steps on an (implicit) energy function (Ho et al., 2020; Song et al., 2021a;b) Informally, one can think of these steps as progressively moving this initial random point closer to the ”data manifold” M .”
- Diffusion Autoencoders: Toward a Meaningful and Decodable Representation (2021, methodology): “In Figure 4d and Figure 1, we show our interpolation results by encoding two input images into (z(1)sem,x 1 T ) and (z 2 sem,x 2 T ), then decode z(t) = (Lerp(z(1)sem, z 2 sem; t),Slerp(x 1 T ,x 2 T ; t)) for various values of t ∈ [0, 1], where linear interpolation is used for zsem and spherical lin”
- Improved Denoising Diffusion Probabilistic Models (2021, methodology): “We did not attempt to calculate NLL using DDIM, since Song et al. (2020a) does not present NLL results or a simple way of estimating likelihood under DDIM.”
- Scalable Diffusion Models with Transformers (2022, background): “Improvements in DDPMs over the past two years have largely been driven by improved sampling techniques [19, 25, 52], most notably classifierfree guidance [21], reformulating diffusion models to predict noise instead of pixels [19] and using cascaded DDPM pipelines where low-resolution base diffusion”
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, methodology): “Sampling such as the ancestral sampler [28] and DDIM [67] start from pure noise z1 ∼ N (0, I) and iteratively generate points zt1 , .”
- Scalable Adaptive Computation for Iterative Generation (2022, methodology): “As a concrete application, we consider iterative generation of images and video with denoising diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2020; 2021).”