Potential Flow Generator With L2 Optimal Transport Regularity for Generative Models

By Liu Yang, G. Em Karniadakis. 2019

50 citations recorded by the source corpus.

We propose a potential flow generator with <inline-formula> <tex-math notation="LaTeX">$L_{2}$ </tex-math></inline-formula> optimal transport regularity, which can be easily integrated into a wide range of generative models, including different versions of generative adversarial networks (GANs) and normalizing flow models. With only a slight augmentation to the original generator loss functions, our generator not only tries to transport the input distribution to the target one but also aims to find the one with minimum <inline-formula> <tex-math notation="LaTeX">$L_{2}$ </tex-math></inline-formula> transport cost. We show the effectiveness of our method in several 2-D problems and illustrate the concept of “proximity” due to the <inline-formula> <tex-math notation="LaTeX">$L_{2}$ </tex-math></inline-formula> optimal transport regularity. Subsequently, we demonstrate the effectiveness of the potential flow generator in image translation tasks with unpaired training data from the MNIST data set and the CelebA data set with a comparison against vanilla Wasserstein GAN with gradient penalty (WGAN-GP) and CycleGAN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Flow Matching for Generative Modeling (2022, neutral): “Furthermore, the energy demand for training large deep learning models is increasing at a rapid pace (Amodei et al., 2018; Thompson et al., 2020), focusing on methods that are able to train using less gradient updates / image throughput can lead to significant time and energy savings.”
How to train your neural ODE (2020, background): “Yang & Karniadakis (2019) continue along these lines, and define an optimal transport again as a scalar potential gradient.”
An introduction to deep generative modeling (2021, background): “There is some numerical evidence that penalizing violations of the Hamilton-Jacobi-Bellman (HJB) equations, which are the necessary and sufficient first-order optimality conditions of (19), improves the practical performance [47, 34].”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “MongeAmp ere Flow [255] and Potential Flow Generators [248] parameterise a potential function satisfying the MongeAmp ere equation [21], [235] with a neural network.”
Stochastic Interpolants: A Unifying Framework for Flows and Diffusions (2023, methodology): “Optimal transport perspectives in flow-based methods have primarily been employed as a regularization penalty [51, 34, 14, 46] or via imposing structure on the parameterization itself [19, 50].”
OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport (2020, methodology): “OT-Flow Problem In summary, the regularized problem solved in OT-Flow is combining aspects from Zhang, E, and Wang (2018), Grath-wohl et al. (2019), Yang and Karniadakis (2020), and Finlay et al. (2020) (Tab.”
B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data (2020, methodology): “In general, using DNF to sample from a target distribution ν consists of the following three steps [20]: 1.”