Noise Estimation for Generative Diffusion Models

By Robin San Roman, Eliya Nachmani, Lior Wolf. 2021

126 citations recorded by the source corpus.

Generative diffusion models have emerged as leading models in speech and image generation. However, in order to perform well with a small number of denoising steps, a costly tuning of the set of noise parameters is needed. In this work, we present a simple and versatile learning scheme that can step-by-step adjust those noise parameters, for any given number of steps, while the previous work needs to retune for each number separately. Furthermore, without modifying the weights of the diffusion model, we are able to significantly improve the synthesis results, for a small number of steps. Our approach comes at a negligible computation cost.

Subfields: Artificial Intelligence, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Generative Models

In Signal Processing

Citations in the corpus

Score-based Generative Modeling in Latent Space (2021, background): “[39] add an adversarial objective, [5] introduce non-Markovian diffusion processes that allow to trade off synthesis speed, quality, and sample diversity, [40] learn a sequence of conditional energy-based models for denoising, [41] distill the iterative sampling process into single shot synthesis, a”
On Fast Sampling of Diffusion Probabilistic Models (2021, background): “, 2020b), and there have been a series of research extending and improving diffusion models (Song et al., 2020b; Gao et al., 2020; Dhariwal & Nichol, 2021; San-Roman et al., 2021; Meng et al., 2021).”
DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps (2022, background): “Samplers that need extra training or optimization include knowledge distillation [13, 14], learning the noise level or variance [15, 16, 33], and learning the noise schedule or sample trajectory [17, 18].”
Learning to Efficiently Sample from Diffusion Probabilistic Models (2021, positive): “…further by Song and Kingma [2021], where other score-matching techniques (e.g., sliced score matching, Song et al. et al., 2020a, Chen et al., 2021, Nichol and Dhariwal, 2021, San-Roman et al., 2021, Kong and Ping, 2021, Jolicoeur-Martineau et al., 2021] has also guided our research efforts.”
ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech (2022, background): “As a blossoming class of generative models, denoising diffusion probabilistic models (DDPMs) have emerged to prove their capability to achieve leading performances in both image and audio synthesis [2, 5, 20, 41], while several challenges remain for industrial deployment: 1) The most dominant diffus”
Bilateral Denoising Diffusion Models (2021, positive): “The same network architecture was used for the NE approach for estimating α 2 t and was shown comparatively better than the ConvTASNet used in the original paper [38].”
Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme (2021, background): “Although several attempts have been recently made to reduce the number of DPM inference steps (Song et al., 2021a; San-Roman et al., 2021; Watson et al., 2021; Kong & Ping, 2021; Chen et al., 2021a), most of them apply to some particular types of DPMs.”
Adding Conditional Control to Text-to-Image Diffusion Models (2023, methodology): “Image diffusion methods can directly use pixel colors as training data, and in that case, researches often consider strategies to save computation powers when handling high-resolution images [53, 50, 26], or directly use pyramid-based or multiple-stage methods [18, 43].”
Deblurring via Stochastic Refinement (2021, background): “As such, many recent works have explored alternative sampling strategies that reduce the number of sampling steps [29, 33, 37, 57, 61, 69].”
Score-Based Generative Modeling with Critically-Damped Langevin Diffusion (2021, background): “Recently, techniques to accelerate sampling from pre-trained SGMs have been proposed (San-Roman et al., 2021; Watson et al., 2021; Kong & Ping, 2021; Song et al., 2021a).”
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs (2021, methodology): “To alleviate this issue, multiple methods have been proposed, including knowledge distillation (Luhman & Luhman, 2021), learning an adaptive noise schedule (San-Roman et al., 2021), introducing non-Markovian diffusion processes (Song et al.”
A Survey on Generative Diffusion Models (2022, background): “Noise Scale Design FastDPM [97]/Improved DDPM [59] VDM [98]/[99]”
DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models (2022, unknown): “Samplers based on extra training include model distillation (Salimans & Ho, 2022; Luhman & Luhman, 2021), learning reverse process variances (San-Roman et al., 2021; Nichol & Dhariwal, 2021; Bao et al., 2022a), and learning sampling steps (Lam et al.”
Itô-Taylor Sampling Scheme for Denoising Diffusion Probabilistic Models using Ideal Derivatives (2021, background): “In addition, there are also some studies aimed at improving the efficiency of the DDPM sampling, including (Jolicoeur-Martineau et al., 2021; Kong & Ping, 2021; San-Roman et al., 2021; Watson et al., 2021).”
Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling (2021, methodology): “DSB complements recent techniques to speed up existing SGM methods that rely on either different noise schedules (Nichol and Dhariwal, 2021; San-Roman et al., 2021; Watson et al., 2021), alternative discretizations (Jolicoeur-Martineau et al.”
Diffusion-GAN: Training GANs with Diffusion (2022, methodology): “For this reason, a variety of methods have been proposed to reduce the generation cost of diffusion-based generative models (Kong and Ping, 2021; Luhman and Luhman, 2021; Pandey et al., 2022; San-Roman et al., 2021; Song et al., 2021a; Xiao et al., 2021; Zheng et al., 2022).”