Bespoke Solvers for Generative Flow Models

By Neta Shaul, Juan Camilo Pérez, Ricky T. Q. Chen, Ali K. Thabet, Albert Pumarola, Yaron Lipman. 2023

43 citations recorded by the source corpus.

Diffusion or flow-based models are powerful generative paradigms that are notoriously hard to sample as samples are defined as solutions to high-dimensional Ordinary or Stochastic Differential Equations (ODEs/SDEs) which require a large Number of Function Evaluations (NFE) to approximate well. Existing methods to alleviate the costly sampling process include model distillation and designing dedicated ODE solvers. However, distillation is costly to train and sometimes can deteriorate quality, while dedicated solvers still require relatively large NFE to produce high quality samples. In this paper we introduce"Bespoke solvers", a novel framework for constructing custom ODE solvers tailored to the ODE of a given pre-trained flow model. Our approach optimizes an order consistent and parameter-efficient solver (e.g., with 80 learnable parameters), is trained for roughly 1% of the GPU time required for training the pre-trained model, and significantly improves approximation and generation quality compared to dedicated solvers. For example, a Bespoke solver for a CIFAR10 model produces samples with Fr\'echet Inception Distance (FID) of 2.73 with 10 NFE, and gets to 1% of the Ground Truth (GT) FID (2.59) for this model with only 20 NFE. On the more challenging ImageNet-64$\times$64, Bespoke samples at 2.2 FID with 10 NFE, and gets within 2% of GT FID (1.71) with 20 NFE.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Apoorv Vyas
Bowen Shi
Matt Le
Andros Tjandra
Yi-Chiao Wu
Baishan Guo
Jiemin Zhang
Xinyue Zhang

Citations in the corpus

Audiobox: Unified Audio Generation with Natural Language Prompts (2023, methodology): “To improve the inference speed at the extreme low NFE regime (i.e., 4), we adopt Bespoke Solvers Shaul et al. (2023) to recover similar sample quality as the original model with a much lower NFE.”
Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation (2024, unknown): “Naturally, a large body of work focuses on speeding up the sampling of diffusion models — both via improved samplers [55, 11, 66, 51] and distilled models that are trained to match the sample quality of their teacher models in fewer steps [44, 35, 34, 58].”