SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers

By Nanye Ma, Mark Goldstein, Michael S. Albergo, Nicholas Matthew Boffi, Eric Vanden-Eijnden, Saining Xie. European Conference on Computer Vision, 2024

672 citations recorded by the source corpus.

We present Scalable Interpolant Transformers (SiT), a family of generative models built on the backbone of Diffusion Transformers (DiT). The interpolant framework, which allows for connecting two distributions in a more flexible way than standard diffusion models, makes possible a modular study of various design choices impacting generative models built on dynamical transport: learning in discrete or continuous time, the objective function, the interpolant that connects the distributions, and deterministic or stochastic sampling. By carefully introducing the above ingredients, SiT surpasses DiT uniformly across model sizes on the conditional ImageNet 256x256 and 512x512 benchmark using the exact same model structure, number of parameters, and GFLOPs. By exploring various diffusion coefficients, which can be tuned separately from learning, SiT achieves an FID-50K score of 2.06 and 2.62, respectively.

View Paper: https://www.semanticscholar.org/paper/1eac5d12f30697aa74d66f4026fb662c5d51bd43

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Enze Xie
2. Junsong Chen
3. Chongjian Ge
4. Zhenguo Li
5. Lewei Yao
6. Ping Luo
7. Yue Wu
8. Zhongdao Wang

Citations in the corpus

• Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (2025, unknown): “(3)”
• Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (2024, unknown): “We strictly follow the architecture used in DiT (Peebles & Xie, 2023) and SiT (Ma et al., 2024a).”
• Mean Flows for One-step Generative Modeling (2025, unknown): “Since its introduction, Flow Matching has seen widespread adoption in modern generative modeling [10, 33, 35].”
• PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation (2024, methodology): “Subsequent works, including DiffiT [16], SiT [29], and FiT [27], have improved upon DiT’s architecture, while [13, 54] enhance training efficiency through masked modeling techniques.”