Elucidating the Design Space of Diffusion-Based Generative Models

By Tero Karras, Miika Aittala, Timo Aila, Samuli Laine. 2022

3,661 citations recorded by the source corpus.

We argue that the theory and practice of diffusion-based generative models are currently unnecessarily convoluted and seek to remedy the situation by presenting a design space that clearly separates the concrete design choices. This lets us identify several changes to both the sampling and training processes, as well as preconditioning of the score networks. Together, our improvements yield new state-of-the-art FID of 1.79 for CIFAR-10 in a class-conditional setting and 1.97 in an unconditional setting, with much faster sampling (35 network evaluations per image) than prior designs. To further demonstrate their modular nature, we show that our design changes dramatically improve both the efficiency and quality obtainable with pre-trained score networks from previous work, including improving the FID of a previously trained ImageNet-64 model from 2.07 to near-SOTA 1.55, and after re-training with our proposed improvements to a new SOTA of 1.36.

Subfields: Biophysics, Computer Vision and Pattern Recognition, Generative Models, Model Architectures, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

  1. Prafulla Dhariwal
  2. Yang Song
  3. Jonathan Ho
  4. Diederik P. Kingma
  5. Alex Nichol
  6. Arash Vahdat
  7. Ruiqi Gao
  8. Jiaming Song

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Ming-Hsuan Yang
  3. Dacheng Tao
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Andrea Vedaldi
  8. Chen Change Loy

In Generative Models

  1. Arash Vahdat
  2. Stefano Ermon
  3. Karsten Kreis
  4. Richard Zhang
  5. Eli Shechtman
  6. Jun-Yan Zhu
  7. Jiaming Song
  8. Prafulla Dhariwal

In Model Architectures

  1. Shuicheng Yan
  2. Luc Van Gool
  3. Diederik P. Kingma
  4. Pan Zhou
  5. Quoc V. Le
  6. Ruiqi Gao
  7. Mohammad Norouzi
  8. Prafulla Dhariwal

In Signal Processing

  1. Jonathan Ho
  2. David J. Fleet
  3. Diederik P. Kingma
  4. Arash Vahdat
  5. Mohammad Norouzi
  6. Yang Song
  7. Prafulla Dhariwal
  8. Stefano Ermon

Citations in the corpus