Improving GANs Using Optimal Transport

By Tim Salimans, Han Zhang, Alec Radford, Dimitris N. Metaxas. 2018

345 citations recorded by the source corpus.

We present Optimal Transport GAN (OT-GAN), a variant of generative adversarial nets minimizing a new metric measuring the distance between the generator distribution and the data distribution. This metric, which we call mini-batch energy distance, combines optimal transport in primal form with an energy distance defined in an adversarially learned feature space, resulting in a highly discriminative distance function with unbiased mini-batch gradients. Experimentally we show OT-GAN to be highly stable when trained with large mini-batches, and we present state-of-the-art results on several popular benchmark problems for image generation.

Subfields: Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

Character-Level Language Modeling with Deeper Self-Attention (2018, methodology): “owing Chung et al. (2015), and as in text8, we split the data into 90M, 5M and 5M for training, dev and test respectively. Training Compared to most models based on transformers (Vaswani et al. 2017; Salimans et al. 2018), our model is very deep, with 64 transformer layers and each layer using two a”
Evaluating Lossy Compression Rates of Deep Generative Models (2020, background): “For this reason, there has been much interest in Wasserstein distance as a criterion for generative models, since the measure exploits precisely this metric structure (Arjovsky et al., 2017; Gulrajani et al., 2017; Salimans et al., 2018).”
Large Scale GAN Training for High Fidelity Natural Image Synthesis (2018, background): “One line of work is focused on changing the objective function (Arjovsky et al., 2017; Mao et al., 2016; Lim & Ye, 2017; Bellemare et al., 2017; Salimans et al., 2018) to encourage convergence.”
Autoregressive Quantile Networks for Generative Modeling (2018, background): “Much of the recent research on GANs has been focused on improving stability (Radford et al., 2015; Arjovsky et al., 2017; Daskalakis et al., 2017) and sample diversity (Gul-rajani et al., 2017; Salimans et al., 2016; 2018).”
Cross-Modal Contrastive Learning for Text-to-Image Generation (2021, background): “A large amount of work has focused on designing the adversarial objective to improve training [12, 1, 31, 47, 29, 54].”
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks (2017, background): “Compared with other deep generative models, Generative Adversarial Networks (GANs) [6] have shown great performance for generating sharper samples [17, 3, 23, 13, 10, 35, 24, 34, 39, 40].”
GAIT: A Geometric Approach to Information Theory (2019, methodology): “Two main reasons for this transition are (i) the need to deal with degenerate distributions (which might have density only over a low dimensional manifold) as is the case in the training of generative models (Goodfellow et al., 2014; Arjovsky et al., 2017; Salimans et al., 2018); and (ii) the develo”
Self-Attention Generative Adversarial Networks (2018, background): “…works have attempted to stabilize the GAN training dynamics and improve the sample diversity by designing new network architectures [22, 37, 10], modifying the learning objectives and dynamics [1, 27, 15, 3, 39], adding regularization methods [7, 16] and introducing heuristic tricks [26, 19].”
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks (2017, methodology): “In order to stabilize the GANs’ training process and improve sample diversity, several methods tried to address the challenges by proposing new network architectures [32], introducing heuristic tricks [40] or modifying the learning objectives [1], [3], [5].”
A Spectral Energy Distance for Parallel Speech Synthesis (2020, positive): “As explained by Sejdinovic et al. [30], Assuming that k ( x , x ) = k ( y , y ) = c with c being a constant, equation (3) can equivalently be expressed in terms of this distance: which is known as the generalized energy distance [GED; see e.g. 30, 31, 28].”
Learning Generative Models with Sinkhorn Divergences (2017, methodology): “Shortly after the submission of this work, we came across the recent work by [30] which shares several ideas with our method.”
Graph Optimal Transport for Cross-Domain Alignment (2020, background): “There are also studies adopting WD in Generative Adversarial Network (GAN) (Goodfellow et al., 2014; Salimans et al., 2018; Chen et al., 2018; Mroueh et al., 2018; Zhang et al., 2020) to alleviate the mode-collapse issue.”
Discriminator optimal transport (2019, background): “Latent space OT itself is not a novel idea [27, 28], but there seems to be no literature using trained G and D, to the best of our knowledge.”
TinyGAN: Distilling BigGAN for Conditional Image Generation (2020, background): “Several works [12,13,14,15,16] have aimed to tackle such problems.”
Interpolating between Optimal Transport and MMD using Sinkhorn Divergences (2018, background): “…from computer graphics (Bonneel et al., 2016) to supervised learning (Frogner et al., 2015), unsupervised density ﬁtting (Bassetti et al., 2006) and generative model learning (Montavon et al., 2016; Arjovsky et al., 2017; Salimans et al., 2018; Genevay et al., 2018; Sanjabi et al., 2018).”
A Probabilistic U-Net for Segmentation of Ambiguous Images (2018, methodology): “To do so, we use the generalized energy distance [26, 27, 28], which leverages distances between observations:”
Learning Implicit Generative Models with the Method of Learned Moments (2018, background): “For clarity, consider the inner maximization step of the Wasserstein GAN: Under the assumption that the generator at θ ∗ is sufﬁ-ciently expressive to model the data distribution, then While MoLM optimizes a directional Jacobian of many moments, adversarial training optimizes a directional derivativ”
UNITER: UNiversal Image-TExt Representation Learning (2019, methodology): “For VQA, VCR, NLVR 2 , Visual Entailment and Image-Text Retrieval, we extract the joint embedding of the input image-text pairs via a multi-layer perceptron (MLP) from the representation of the [CLS] token.”