Matching Normalizing Flows and Probability Paths on Manifolds

By Heli Ben-Hamu, Samuel Cohen, Joey Bose, Brandon Amos, Maximilian Nickel, Aditya Grover, Ricky T. Q. Chen, Yaron Lipman. 2022

64 citations recorded by the source corpus.

Continuous Normalizing Flows (CNFs) are a class of generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE). We propose to train CNFs on manifolds by minimizing probability path divergence (PPD), a novel family of divergences between the probability density path generated by the CNF and a target probability density path. PPD is formulated using a logarithmic mass conservation formula which is a linear first order partial differential equation relating the log target probabilities and the CNF's defining vector field. PPD has several key benefits over existing methods: it sidesteps the need to solve an ODE per iteration, readily applies to manifold data, scales to high dimensions, and is compatible with a large family of target paths interpolating pure noise and data in finite time. Theoretically, PPD is shown to bound classical probability divergences. Empirically, we show that CNFs learned by minimizing PPD achieve state-of-the-art results in likelihoods and sample quality on existing low-dimensional manifold benchmarks, and is the first example of a generative model to scale to moderately high dimensional manifolds.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Building Normalizing Flows with Stochastic Interpolants (2022, unknown): “, 2021) and biased optimization (Ben-Hamu et al., 2022), and are limited to manifolds.”
SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers (2024, methodology): “Velocity field parameterizations using the Linear interpolant were also studied in [39, 42], and were generalized to the manifold setting in [6].”
Stochastic Interpolants: A Unifying Framework for Flows and Diffusions (2023, background): “At the heart of progress is the idea to apply ordinary or stochastic differential equations (ODE/SDE) to continuously transform samples from a base probability density function (PDF) ρ0 into samples from a target density ρ1 (or vice-versa), and the realization that inference over the velocity field ”