Fast Sampling of Diffusion Models with Exponential Integrator

By Qinsheng Zhang, Yongxin Chen. 2022

629 citations recorded by the source corpus.

The past few years have witnessed the great success of Diffusion models~(DMs) in generating high-fidelity samples in generative modeling tasks. A major limitation of the DM is its notoriously slow sampling procedure which normally requires hundreds to thousands of time discretization steps of the learned diffusion process to reach the desired accuracy. Our goal is to develop a fast sampling method for DMs with a much less number of steps while retaining high sample quality. To this end, we systematically analyze the sampling procedure in DMs and identify key factors that affect the sample quality, among which the method of discretization is most crucial. By carefully examining the learned diffusion process, we propose Diffusion Exponential Integrator Sampler~(DEIS). It is based on the Exponential Integrator designed for discretizing ordinary differential equations (ODEs) and leverages a semilinear structure of the learned diffusion process to reduce the discretization error. The proposed method can be applied to any DMs and can generate high-fidelity samples in as few as 10 steps. In our experiments, it takes about 3 minutes on one A6000 GPU to generate $50k$ images from CIFAR10. Moreover, by directly using pre-trained DMs, we achieve the state-of-art sampling performance when the number of score function evaluation~(NFE) is limited, e.g., 4.17 FID with 10 NFEs, 3.37 FID, and 9.74 IS with only 15 NFEs on CIFAR10. Code is available at https://github.com/qsh-zh/deis

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

In Numerical Methods & Scientific Computing

Citations in the corpus

Flow Matching for Generative Modeling (2022, background): “However, the restriction to simple diffusion processes leads to a rather confined space of sampling probability paths, resulting in very long training times and the need to adopt specialized methods ( e.g. , Song et al. (2020a); Zhang & Chen (2022)) for efficient sampling.”
From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (2024, positive): “There are many possible sampling methods [30, 51, 73, 101] to compute the sample x t − 1 in the next time step from the current one x t based on the predicted noise ϵ θ ( x t , t ) .”
UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models (2023, methodology): “Recently, there have been increasing efforts to accelerate the sampling of DPMs [31, 28, 34, 25, 40].”
ModelScope Text-to-Video Technical Report (2023, background): “To solve the first obstacle, research efforts focused on improving the sampling efficiency by learning-free sampling [54, 55, 35, 69, 37, 69] and learning based sampling [60, 70, 49].”
I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models (2023, background): “For this purpose, some studies aim to reduce the number of denoising steps in the model by improving sampling efficiency, such as learning-free sampling [24,42,43,56,56] and learning based sampling [37, 48, 60].”
Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation (2024, positive): “Naturally, a large body of work focuses on speeding up the sampling of diffusion models — both via improved samplers [55, 11, 66, 51] and distilled models that are trained to match the sample quality of their teacher models in fewer steps [44, 35, 34, 58].”
Adversarial Diffusion Distillation (2023, methodology): “In addition to utilizing faster samplers for diffusion models [8, 37, 64, 74], there is a growing body of research on model distillation such as progressive distillation [56] and guidance distillation [43].”
Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models (2023, neutral): “15 [108] Qinsheng Zhang and Yongxin Chen.”
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow (2022, background): “[94] proposes a diffusion exponential integrator sampler for fast sampling of diffusion models.”
A Continuous Time Framework for Discrete Denoising Models (2022, background): “Our work has opened the door to applying the work improving sampling speed on continuous data [14, 15, 16, 17] to discrete data problems too.”
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data (2024, neutral): “Intrinsically, this is because such models characterize the joint distribution by the chain rule of probability, motivating research on developing other types of generative models for text.”
PhysDiff: Physics-Guided Human Motion Diffusion Model (2022, methodology): “These models are trained via denoising autoencoder objectives that can be interpreted as score matching [84], and generate samples via an iterative denoising procedure that may use stochastic updates [5, 6, 15, 98] which solve stochastic differential equations (SDEs) or deterministic updates [14,30,”
DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models (2022, positive): “…DPM-Solver (order = 2) (Lu et al., 2022) PNDM (order = 2) (Liu et al., 2022b) DPM-Solver-3 (order = 3) (Lu et al., 2022) DEIS-1 (order = 2) (Zhang & Chen, 2022) † DDIM (thresholding) (Saharia et al., 2022b) DEIS-2 (order = 3) (Zhang & Chen, 2022) DPM-Solver++ (order = 2) ( ours ) Figure 1:…”
Bespoke Solvers for Generative Flow Models (2023, methodology): “Lu et al. (2022a); Zhang & Chen (2022) utilize the semi-linear structure of the score/ ϵ -based sampling ODE to adopt a method of exponential integrators.”
A Survey on Generative Diffusion Models (2022, methodology): “Later works [29], [47] interpret DDIM as a product of applying an exponential integrator on the ODE of Variance Preserving (VP) diffusion [11].”