Monge-Ampère Flow for Generative Modeling

By Linfeng Zhang, E. Weinan, Lei Wang. 2018

76 citations recorded by the source corpus.

We present a deep generative model, named Monge-Ampere flow, which builds on continuous-time gradient flow arising from the Monge-Ampere equation in optimal transport theory. The generative map from the latent space to the data space follows a dynamical system, where a learnable potential function guides a compressible fluid to flow towards the target density distribution. Training of the model amounts to solving an optimal control problem. The Monge-Ampere flow has tractable likelihoods and supports efficient sampling and inference. One can easily impose symmetry constraints in the generative model by designing suitable scalar potential functions. We apply the approach to unsupervised density estimation of the MNIST dataset and variational calculation of the two-dimensional Ising model at the critical point. This approach brings insights and techniques from Monge-Ampere equation, optimal transport, and fluid dynamics into reversible flow-based generative models.

Subfields: Applied Mathematics, Geometry and Topology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

1. Samy Wu Fung
2. Yang Long
3. Bo Dai
4. Eldad Haber
5. Arthur Gretton
6. Dale Schuurmans
7. Le Song
8. Sam Bond-Taylor

In Geometry and Topology

1. Samy Wu Fung
2. Eldad Haber
3. Arthur Gretton
4. Le Song
5. Hanjun Dai
6. Derek Onken
7. Lars Ruthotto

Citations in the corpus

• Equivariant Flows: exact likelihood generative learning for symmetric densities (2020, background): “Gradient flows (not necessarily G-equivariant) can map any ρ onto any ρ′ over Rn as long as both densities do not vanish (Benamou & Brenier, 2000; McCann, 2001) and have been discussed in the context of density estimation (Zhang et al., 2018; Papamakarios et al., 2019).”
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “A simple example of this is the affine coupling layer [42], yðdþ1:DÞ ¼ xðdþ1:DÞ expðfsðxð1:dÞÞÞ þ fmðxð1:dÞÞ; (32) which has a simple Jacobian determinant and can be trivially rearranged to obtain a definition of xðdþ1:DÞ in terms of TABLE 3 Normalizing Flow Layers: Represents Elementwise Multiplica”
• OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport (2020, methodology): “OT-Flow Problem In summary, the regularized problem solved in OT-Flow is combining aspects from Zhang, E, and Wang (2018), Grath-wohl et al. (2019), Yang and Karniadakis (2020), and Finlay et al. (2020) (Tab.”
• An introduction to deep generative modeling (2021, methodology): “Several approaches that add transport costs to the CNF problem have been proposed recently [48, 47, 14, 30, 34].”
• Exponential Family Estimation via Adversarial Dynamics Embedding (2019, methodology): “We noticed that Zhang et al. (2018) also exploits the Monge-Ampère equation to design the flow-based model for unsupervised learning.”