On Fast Sampling of Diffusion Probabilistic Models

By Zhifeng Kong, Wei Ping. 2021

228 citations recorded by the source corpus.

In this work, we propose FastDPM, a unified framework for fast sampling in diffusion probabilistic models. FastDPM generalizes previous methods and gives rise to new algorithms with improved sample quality. We systematically investigate the fast sampling methods under this framework across different domains, on different datasets, and with different amount of conditional information provided for generation. We find the performance of a particular method depends on data domains (e.g., image or audio), the trade-off between sampling speed and sample quality, and the amount of conditional information. We further provide insights and recipes on the choice of methods for practitioners.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Statistics and Probability, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Statistics and Probability

In Generative Models

Citations in the corpus

High-Resolution Image Synthesis with Latent Diffusion Models (2021, background): “While the former can be partially adressed by advanced sampling strategies [46, 73, 81] and hierarchical approaches [30, 90], training on high-resolution image data always requires to calculate expensive gradients.”
DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps (2022, methodology): “3.4 for using DPM-Solver in discrete-time DPMs, and then compare DPM-Solver with other discrete-time training-free samplers, including DDPM [2], DDIM [19], Analytic-DDPM [21], Analytic-DDIM [21], PNDM [22], FastDPM [38] and Itô-Taylor [24].”
A Survey on Neural Speech Synthesis (2021, background): “Thus, a lot of works on diffusion models [307, 181, 378, 171] are investigating how to reduce inference time while maintaining generation quality.”
Learning to Efficiently Sample from Diffusion Probabilistic Models (2021, positive): “…further by Song and Kingma [2021], where other score-matching techniques (e.g., sliced score matching, Song et al. et al., 2020a, Chen et al., 2021, Nichol and Dhariwal, 2021, San-Roman et al., 2021, Kong and Ping, 2021, Jolicoeur-Martineau et al., 2021] has also guided our research efforts.”
Bilateral Denoising Diffusion Models (2021, neutral): “Noise scheduling has been shown signiﬁcant [2, 3, 35] for sampling acceleration [2] and for high-ﬁdelity generation.”
Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme (2021, background): “Although several attempts have been recently made to reduce the number of DPM inference steps (Song et al., 2021a; San-Roman et al., 2021; Watson et al., 2021; Kong & Ping, 2021; Chen et al., 2021a), most of them apply to some particular types of DPMs.”
DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models (2022, unknown): “…be understood as either discretizing diffusion SDEs (Ho et al., 2020; Song et al., 2021b; Jolicoeur-Martineau et al., 2021; Tachibana et al., 2021; Kong & Ping, 2021; Bao et al., 2022b; Zhang et al., 2022) or discretizing diffusion ODEs (Song et al., 2021b;a; Liu et al., 2022b; Zhang & Chen,…”
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior (2021, methodology): “Same as Kong et al. (2021), the network is also conditioned on the discretized Based on the mel-spectrogram condition, we propose leveraging a normalized frame-level energy of the mel-spectrogram for acquiring data-dependent prior, exploiting the fact that the spectral energy contains an exact…”
Adding Conditional Control to Text-to-Image Diffusion Models (2023, methodology): “Image diffusion methods can directly use pixel colors as training data, and in that case, researches often consider strategies to save computation powers when handling high-resolution images [53, 50, 26], or directly use pyramid-based or multiple-stage methods [18, 43].”
VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation (2023, methodology): “Diffusion probabilistic models (DPMs) are a class of deep generative models, which consist of : i) a diffusion process that gradually adds noise to data points, and ii) a denoising process that generates new samples via iterative denoising [14, 18].”
Crystal Diffusion Variational Autoencoder for Periodic Material Generation (2021, background): “There are also many recent works that aim to speed up the sampling process for diffusion models (Nichol & Dhariwal, 2021; Kong & Ping, 2021; Salimans & Ho, 2022).”
Itô-Taylor Sampling Scheme for Denoising Diffusion Probabilistic Models using Ideal Derivatives (2021, background): “In addition, there are also some studies aimed at improving the efﬁciency of the DDPM sampling, including (Jolicoeur-Martineau et al., 2021; Kong & Ping, 2021; San-Roman et al., 2021; Watson et al., 2021).”
SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models (2021, unknown): “Though the diffusion model is relatively slow in inference, it has great potential to generate images much faster, according to recent works [57–59], which focus on boosting the inference speed of diffusion models.”
Structure and Content-Guided Video Synthesis with Diffusion Models (2023, background): “(2) provides a direct way to sample from the trained model, many other sampling methods [52, 24, 22] require only a fraction of the number of diffusion timesteps to achieve good sample quality.”
Subspace Diffusion Generative Models (2022, background): “Other works [13, 3, 10, 22, 17, 11, 2] focus on reducing the number of steps in the discrete-time Markov chain formulation.”