Improving and generalizing flow-based generative models with minibatch optimal transport

By Alexander Tong, Esmeralda S. Whitammer, G. Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Kilian Fatras, Guy Wolf, Yoshua Bengio. Trans. Mach. Learn. Res., 2023

889 citations recorded by the source corpus.

Continuous normalizing flows (CNFs) are an attractive generative modeling technique, but they have been held back by limitations in their simulation-based maximum likelihood training. We introduce the generalized conditional flow matching (CFM) technique, a family of simulation-free training objectives for CNFs. CFM features a stable regression objective like that used to train the stochastic flow in diffusion models but enjoys the efficient inference of deterministic flow models. In contrast to both diffusion models and prior CNF training algorithms, CFM does not require the source distribution to be Gaussian or require evaluation of its density. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference, as evaluated in our experiments. Furthermore, we show that when the true OT plan is available, our OT-CFM method approximates dynamic OT. Training CNFs with CFM improves results on a variety of conditional and unconditional generation tasks, such as inferring single cell dynamics, unsupervised image translation, and Schr\"odinger bridge inference.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/5396c55bee2a2abf2207e1cc5e5ae72c9edef9fa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Signal Processing

Citations in the corpus

Discrete Flow Matching (2024, methodology): “In Continuous FM one can chose several ways to define the probability paths (Lipman et al., 2022; Liu et al., 2022; Albergo and Vanden-Eijnden, 2022; Pooladian et al., 2023; Tong et al., 2023): Denoiser parameterization.”
SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers (2024, background): “[50, 64] compute mini-batch optimal couplings between the Gaussian and data distribution to reduce the transport cost and gradient variance; [4] instead build the coupling by flowing directly from the conditioning variable to the data for image-conditional tasks such as super-resolution and…”