DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

By Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu. 2022

2,469 citations recorded by the source corpus.

Diffusion probabilistic models (DPMs) are emerging powerful generative models. Despite their high-quality generation performance, DPMs still suffer from their slow sampling as they generally need hundreds or thousands of sequential function evaluations (steps) of large neural networks to draw a sample. Sampling from DPMs can be viewed alternatively as solving the corresponding diffusion ordinary differential equations (ODEs). In this work, we propose an exact formulation of the solution of diffusion ODEs. The formulation analytically computes the linear part of the solution, rather than leaving all terms to black-box ODE solvers as adopted in previous works. By applying change-of-variable, the solution can be equivalently simplified to an exponentially weighted integral of the neural network. Based on our formulation, we propose DPM-Solver, a fast dedicated high-order solver for diffusion ODEs with the convergence order guarantee. DPM-Solver is suitable for both discrete-time and continuous-time DPMs without any further training. Experimental results show that DPM-Solver can generate high-quality samples in only 10 to 20 function evaluations on various datasets. We achieve 4.70 FID in 10 function evaluations and 2.87 FID in 20 function evaluations on the CIFAR10 dataset, and a $4\sim 16\times$ speedup compared with previous state-of-the-art training-free samplers on various datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

Citations in the corpus

Elucidating the Design Space of Diffusion-Based Generative Models (2022, background): “Improving the output quality and/or decreasing the computational cost of sampling are common topics in diffusion model research (e.g., [10, 24, 30, 31, 32, 36, 43, 50, 52, 53, 56]).”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Here, we discuss some attractive intersections of continual learning with other topics of the broad AI community: Diffusion Model [30], [93], [278], [391] is a rising state-ofthe-art generative model, which constructs a Markov chain”
StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models (2023, background): “4 with the ancestral DPM-2 solver [54] for fast and diverse sampling as we demand speed more than accuracy.”
Continuous diffusion for categorical data (2022, methodology): “Based on results in the image domain, it may be possible to reduce this further with stochastic samplers 11 , or more advanced sampling algorithms (Lu et al., 2022a,b).”
I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models (2023, methodology): “During inference, we use DDIM [57] and DPM-solver++ [26] by considering the generation efficiency and generation quality at different resolutions.”
Muse: Text-To-Image Generation via Masked Generative Transformers (2023, background): “…Generated Images HighRes Generated Images Step = 4 Step = 2 Step = 8 Step = 4 Step = 2 Step = 6 Step = 12 Step = 14 Step = 16 Step = 8 Step = 18 (Lu et al., 2022) have greatly reduced the sampling steps of diffusion models, but they have not been widely validated in large scale text-to-image…”
PhysDiff: Physics-Guided Human Motion Diffusion Model (2022, background): “…be interpreted as score matching [80], and generate samples via an iterative denoising procedure that may use stochastic updates [6, 5, 15, 94] which solve stochastic differential equations (SDEs) or deterministic updates [74, 43, 41, 93, 30, 14] which solve ordinary differential equations (ODEs).”
Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, methodology): “Numerous methods have been developed to improve diffusion models, either by enhancing empirical performance [166, 217, 221] or by extending the model’s capacity from a theoretical perspective [145, 146, 219, 225, 277].”
Q-Diffusion: Quantizing Diffusion Models (2023, methodology): “clude simulating diffusion process in fewer steps by generalizing it to a non-Markovian process [42], adjusting the variance schedule [31], and leveraging high-order solvers to approximate diffusion generation [25, 1, 27, 28].”
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, background): “To meet the specific needs of different users, Ruiz et al .”
One-Step Diffusion with Distribution Matching Distillation (2023, background): “The ﬁrst type advances fast diffusion samplers [31, 41, 45, 46, 90], which can dramatically reduce the number of sampling steps required by pre-trained diffusion models—from a thousand down to merely 20-50.”
Better Diffusion Models Further Improve Adversarial Training (2023, background): “Later, there are emerging research routines that, to name a few, accelerate sampling inference (Song et al., 2021a; Lu et al., 2022), optimize model parametrization and sampling schedule (Kingma et al.”
DeepCache: Accelerating Diffusion Models for Free (2023, methodology): “Efforts to accelerate these models have focused on two main strategies: reducing the number of sampling steps, as seen in approaches [36, 41, 54, 60], and decreasing the model inference overhead per step through methods like model pruning, distillation, and quantization [10, 13, 23].”
MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation (2023, background): “The number of denoising steps, and the parameterization of the transformation varies among different works (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2020; Lu et al., 2022a;b; Liu et al., 2022).”
DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization (2023, background): “…have also drawn connections to stochastic differential equations (SDEs) [104] and ordinary differential equations (ODEs) [101] in a continuous time framework, leading to improved sampling algorithms by solving discretized SDEs/ODEs with higher-order solvers [73, 75, 74] or implicit diffusion [101].”
LayoutDiffusion: Controllable Diffusion Model for Layout-to-Image Generation (2023, methodology): “Specifically, we adapt DPM-solver [21] for the conditional classifier-free sampling, a fast dedicated high-order solver for diffusion ODEs [36] with the convergence order guarantee, to accelerate the conditional sampling speed.”