Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling

By Valentin De Bortoli, James Thornton, Jeremy Heng, Arnaud Doucet. 2021

746 citations recorded by the source corpus.

Progressively applying Gaussian noise transforms complex data distributions to approximately Gaussian. Reversing this dynamic defines a generative model. When the forward noising process is given by a Stochastic Differential Equation (SDE), Song et al. (2021) demonstrate how the time inhomogeneous drift of the associated reverse-time SDE may be estimated using score-matching. A limitation of this approach is that the forward-time SDE must be run for a sufficiently long time for the final distribution to be approximately Gaussian. In contrast, solving the Schr\"odinger Bridge problem (SB), i.e. an entropy-regularized optimal transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time. We present Diffusion SB (DSB), an original approximation of the Iterative Proportional Fitting (IPF) procedure to solve the SB problem, and provide theoretical analysis along with generative modeling experiments. The first DSB iteration recovers the methodology proposed by Song et al. (2021), with the flexibility of using shorter time intervals, as subsequent DSB iterations reduce the discrepancy between the final-time marginal of the forward (resp. backward) SDE with respect to the prior (resp. data) distribution. Beyond generative modeling, DSB offers a widely applicable computational optimal transport tool as the continuous state-space analogue of the popular Sinkhorn algorithm (Cuturi, 2013).

Subfields: Biomedical Engineering, Cognitive Neuroscience, Control and Systems Engineering, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Cognitive Neuroscience

In Control and Systems Engineering

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, background): “[13] Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet.”
Flow Matching for Generative Modeling (2022, neutral): “In an another line of works, (De Bortoli et al., 2021; Wang et al., 2021; Peluchetti, 2021) proposed finite time diffusion constructions via diffusion bridges theory resolving the approximation error incurred by infinite time denoising constructions.”
Riemannian Score-Based Generative Modeling (2022, methodology): “To circumvent this problem, one can consider a process whose initial and terminal distribution are pinned down using Schrödinger bridges (Schrödinger, 1932; Léonard, 2012a; Chen et al., 2016; De Bortoli et al., 2021; Vargas et al., 2021).”
A Continuous Time Framework for Discrete Denoising Models (2022, background): “The continuous time interpretation has also enabled the derivation of interesting theoretical properties such as error bounds [19] in continuous state spaces.”
Non-Denoising Forward-Time Diffusions (2023, methodology): “De Bortoli et al. (2021, Theorem 1) links these approximations to the total variation distance between the distribution of the generated samples from (iv) and P D .”
Fast Sampling of Diffusion Models with Exponential Integrator (2022, background): “Another strategy to make the forward diffusion nonlinear and trainable (Zhang & Chen, 2021; Vargas et al., 2021; De Bortoli et al., 2021; Wang et al., 2021; Chen et al., 2021a) in the spirit of Schr¨odinger bridge (Chen et al., 2021b).”
Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion (2023, background): “…Differential Equations (SDE) or Ordinary Differential Equations (ODE) substantially reduce the required Number of Function Evaluations (NFE), further improvements are challenging due to the intrinsic discretization error present in all differential equation solvers (De Bortoli et al., 2021).”
SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers (2024, background): “Finally, various work considers learning a stochastic bridge connecting two arbitrary distributions [18, 41, 49, 56].”
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow (2022, background): “Another body of works [87, 11, 7, 82] leverages Schrodinger bridges (SB) as an alternative approach to learning diffusion generative models.”
Stochastic Interpolants: A Unifying Framework for Flows and Diffusions (2023, background): “A variety of recent works have formulated the Schrödinger problem in the context of a learnable diffusion [5, 43].”
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting (2022, methodology): “Imagen Editor is a diffusion-based model [6] fine-tuned from Imagen [36] for editing.”
Improving and generalizing flow-based generative models with minibatch optimal transport (2023, neutral): “…size needed to approximate the true dynamic optimal transport paths will vary with dataset (for example in the moon-8gaussian case we need a larger batch size) it is still somewhat surprising that such small batches are needed as this is less than 0.5% of the entire 10k point dataset per batch.”
Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, background): “Recent advances in Schrödinger bridges and optimal transport [31, 44, 46, 212, 218] provide promising alternative solutions, suggesting new formulations for diffusion models that are capable of converging to a specified prior distribution in finite time.”