Building Normalizing Flows with Stochastic Interpolants

By Michael S. Albergo, Eric Vanden-Eijnden. 2022

980 citations recorded by the source corpus.

A generative model based on a continuous-time normalizing flow between any pair of base and target probability densities is proposed. The velocity field of this flow is inferred from the probability current of a time-dependent density that interpolates between the base and the target in finite time. Unlike conventional normalizing flow inference methods based the maximum likelihood principle, which require costly backpropagation through ODE solvers, our interpolant approach leads to a simple quadratic loss for the velocity itself which is expressed in terms of expectations that are readily amenable to empirical estimation. The flow can be used to generate samples from either the base or target, and to estimate the likelihood at any time along the interpolant. In addition, the flow can be optimized to minimize the path length of the interpolant density, thereby paving the way for building optimal transport maps. In situations where the base is a Gaussian density, we also show that the velocity of our normalizing flow can also be used to construct a diffusion model to sample the target as well as estimate its score. However, our approach shows that we can bypass this diffusion completely and work at the level of the probability flow with greater simplicity, opening an avenue for methods based solely on ordinary differential equations as an alternative to those based on stochastic differential equations. Benchmarking on density estimation tasks illustrates that the learned flow can match and surpass conventional continuous flows at a fraction of the cost, and compares well with diffusions on image generation on CIFAR-10 and ImageNet $32\times32$. The method scales ab-initio ODE flows to previously unreachable image resolutions, demonstrated up to $128\times128$.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Statistical and Nonlinear Physics

Citations in the corpus

Flow Matching for Generative Modeling (2022, neutral): “Lastly, concurrently to our work (Liu et al., 2022; Albergo & Vanden-Eijnden, 2022) arrived at similar conditional objectives for simulation-free training of CNFs, while Neklyudov et al. (2023) derived an implicit objective when u t is assumed to be a gradient field.”
Improving and generalizing flow-based generative models with minibatch optimal transport (2023, neutral): “%) computational overhead during training.”
Mean Flows for One-step Generative Modeling (2025, positive): “Flow Matching [28, 1, 30] provides an intuitive and conceptually simple framework for constructing flow paths that transport one distribution to another.”
FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space (2025, neutral): “Adversarial Diffusion Distillation Sampling of a flow matching model obtained by optimizing Equation (3) typically involves solving an ordinary or stochastic differential equation [30, 1], using 50–250 guided [16] network evaluations.”
DanceGRPO: Unleashing GRPO on Visual Generation (2025, neutral): “Inspired by [32–34], we introduce an SDE case during the reverse process as follows: where ε t also introduces the stochasticity during sampling.”
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation (2023, background): “The whole fine-tuning process to yield InstaFlow is pure supervised learning and costs only 199 A100 GPU days.”
SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers (2024, methodology): “Error bounds on the ODE are also explored in [7, 15], in addition to the Wasserstein bounds provided in [1].”
Stochastic Interpolants: A Unifying Framework for Flows and Diffusions (2023, methodology): “In order to satisfy the desirable characteristics outlined in the previous paragraph, here we develop a framework for generative modeling based on the method proposed in [1], with two modifications – one that improves the baseline performance, and one that enables us to quantify explicitly and explo”