InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation

By Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, Qiang Liu. International Conference on Learning Representations, 2023

407 citations recorded by the source corpus.

Diffusion models have revolutionized text-to-image generation with its exceptional quality and creativity. However, its multi-step sampling process is known to be slow, often requiring tens of inference steps to obtain satisfactory results. Previous attempts to improve its sampling speed and reduce computational costs through distillation have been unsuccessful in achieving a functional one-step model. In this paper, we explore a recent method called Rectified Flow, which, thus far, has only been applied to small datasets. The core of Rectified Flow lies in its \emph{reflow} procedure, which straightens the trajectories of probability flows, refines the coupling between noises and images, and facilitates the distillation process with student models. We propose a novel text-conditioned pipeline to turn Stable Diffusion (SD) into an ultra-fast one-step model, in which we find reflow plays a critical role in improving the assignment between noise and images. Leveraging our new pipeline, we create, to the best of our knowledge, the first one-step diffusion-based text-to-image generator with SD-level image quality, achieving an FID (Frechet Inception Distance) of $23.3$ on MS COCO 2017-5k, surpassing the previous state-of-the-art technique, progressive distillation, by a significant margin ($37.2$ $\rightarrow$ $23.3$ in FID). By utilizing an expanded network with 1.7B parameters, we further improve the FID to $22.4$. We call our one-step models \emph{InstaFlow}. On MS COCO 2014-30k, InstaFlow yields an FID of $13.1$ in just $0.09$ second, the best in $\leq 0.1$ second regime, outperforming the recent StyleGAN-T ($13.9$ in $0.1$ second). Notably, the training of InstaFlow only costs 199 A100 GPU days. Codes and pre-trained models are available at \url{github.com/gnobitab/InstaFlow}.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/448e7e873a6e520072301b7d9c1d0d9289778418

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference (2023, background): “However, it is computationally intensive (e.g. at least 45 A100 GPUs Days for 2-step inference, estimated in (Liu et al., 2023)).”
Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding (2024, methodology): “Algorithmic Acceleration Recently, various methods have been proposed to reduce the inference steps of diffusion-based text-to-image models [22, 31, 21, 37, 40].”
From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (2024, unknown): “Unlike trajectory-preserving methods [32, 47, 69], DMD provides supervision at the distribution level and offers the unique advantage of allowing different architectural formulations for the teacher and student diffusion models.”
Text-to-3D using Gaussian Splatting (2023, background): “One of the most successful applications of diffusion models lies in text-to-image generation, where they have shown remarkable progress in generating realistic images from text prompts [1, 26, 55].”
Improved Distribution Matching Distillation for Fast Image Synthesis (2024, methodology): “A complementary technique, Instaflow [11] straightens the ODE trajectories, so they are easier to approximate with a one-step student.”
One-Step Diffusion with Distribution Matching Distillation (2023, methodology): “To accelerate sampling speed, previous methods [42, 43, 47, 48, 51, 65, 75, 90, 91] distill the noise image mapping, discovered by the original multi-step diffusion sampling, into a single-pass student network.”
Adversarial Diffusion Distillation (2023, methodology): “InstaFlow [36], and OpenMuse [48].”