Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise

By Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, Tom Goldstein. Neural Information Processing Systems, 2022

440 citations recorded by the source corpus.

Standard diffusion models involve an image transform -- adding Gaussian noise -- and an image restoration operator that inverts this degradation. We observe that the generative behavior of diffusion models is not strongly dependent on the choice of image degradation, and in fact an entire family of generative models can be constructed by varying this choice. Even when using completely deterministic degradations (e.g., blur, masking, and more), the training and test-time update rules that underlie diffusion models can be easily generalized to create generative models. The success of these fully deterministic models calls into question the community's understanding of diffusion models, which relies on noise in either gradient Langevin dynamics or variational inference, and paves the way for generalized diffusion models that invert arbitrary processes. Our code is available at https://github.com/arpitbansal297/Cold-Diffusion-Models

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/525f459f369032e2f2fa3eb1d60da34ab99191bc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, methodology): “This line of research is promising, especially when one has already confirmed the noise type and schedule in a discrete-time setting [9, 21].”
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, background): “Furthermore, cold diffusion [128] proposes a more generalized conclusion that noise can be set to any distribution as the generative behavior of diffusion model is not strongly dependent on the choice of noise distribution.”
A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI (2023, positive): “Compared to the default Gaussian noise in DDPM [24], cold diffusion [3] improves the sampling procedure in the computer vision area and shows impressive generalization capability in a broad family of degradations, e.”
A Survey on Generative Diffusion Models (2022, background): “For instance, Cold Diffusion [90] uses arbitrary image transformations like blurring for the forward process, while [118] applies heat dissipation in pixel space.”
Building Normalizing Flows with Stochastic Interpolants (2022, positive): “Namely, the SDE must evolve for inﬁnite time to connect the distributions, the parameterization of the time steps remains heuristic (Xiao et al., 2022), and the criticality of noise, as well as the score, is not absolutely apparent (Bansal et al., 2022; Lu et al., 2022).”
Universal Guidance for Diffusion Models (2023, background): “Methods from this category require training new diffusion models that accept the prompt as an additional input [2, 9, 18, 27, 29].”