Training GANs with Optimism

By Constantinos Daskalakis, Andrew Ilyas, Vasilis Syrgkanis, Haoyang Zeng. 2017

593 citations recorded by the source corpus.

We address the issue of limit cycling behavior in training Generative Adversarial Networks and propose the use of Optimistic Mirror Decent (OMD) for training Wasserstein GANs. Recent theoretical results have shown that optimistic mirror decent (OMD) can enjoy faster regret rates in the context of zero-sum games. WGANs is exactly a context of solving a zero-sum game with simultaneous no-regret dynamics. Moreover, we show that optimistic mirror decent addresses the limit cycling problem in training WGANs. We formally show that in the case of bi-linear zero-sum games the last iterate of OMD dynamics converges to an equilibrium, in contrast to GD dynamics which are bound to cycle. We also portray the huge qualitative difference between GD and OMD dynamics with toy examples, even when GD is modified with many adaptations proposed in the recent literature, such as gradient penalty or momentum. We apply OMD WGAN training to a bioinformatics problem of generating DNA sequences. We observe that models trained with OMD achieve consistently smaller KL divergence with respect to the true underlying distribution, than models trained with GD variants. Finally, we introduce a new algorithm, Optimistic Adam, which is an optimistic variant of Adam. We apply it to WGAN training on CIFAR10 and observe improved performance in terms of inception score as compared to Adam.

Subfields: Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

Autoregressive Quantile Networks for Generative Modeling (2018, background): “Much of the recent research on GANs has been focused on improving stability (Radford et al., 2015; Arjovsky et al., 2017; Daskalakis et al., 2017) and sample diversity (Gul-rajani et al., 2017; Salimans et al., 2016; 2018).”
Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity (2021, methodology): “However, as pointed out among others by Daskalakis et al. [2018] and Chavdarova et al. [2019], getting last-iterate convergence is important to apply the methods on potentially non-monotone problems such as GANs, where averaging is not appropriate.”
Stochastic Hamiltonian Gradient Methods for Smooth Games (2020, neutral): “2 This behavior is also problematic in the context of applying the method on non-convex problems, where averaging do not necessarily yield a solution (Daskalakis et al., 2018; Abernethy et al., 2019).”
A Closer Look at the Optimization Landscapes of Generative Adversarial Networks (2019, background): “, 2019), optimism (Daskalakis et al., 2018) or gradient penalty based methods (Mescheder et al.”
Reducing Noise in GAN Training with Variance Reduced Extragradient (2019, methodology): “Daskalakis et al. (2018) proposed a method called Optimistic-Adam inspired by game theory.”
Negative Momentum for Improved Game Dynamics (2018, methodology): “Daskalakis et al. (2018) extrapolate the next value of the gradient using previous history and Gidel”
A Variational Inequality Perspective on Generative Adversarial Networks (2018, methodology): “Daskalakis et al. (2018) proposed a method inspired from game theory.”
Agnostic Federated Learning (2019, background): “Recently, Rakhlin and Sridharan (2013) and Daskalakis et al. (2017) gave an optimistic gradient descent algorithm for minimax optimizations.”
Diagnosing Bottlenecks in Deep Q-learning Algorithms (2019, methodology): “Our results provided in the main paper and here don’t particularly assume any other tricks like Optimistic Gradient (Daskalakis et al., 2018), using exponential moving average of the parameters (Yazıcı et al., 2019).”
Machine Learning with Membership Privacy using Adversarial Regularization (2018, background): “The recent advances in machine learning, notably the developments of generative adversarial networks [3, 12, 21], have introduced new algorithms for solving min-max games while training a complex (deep neural network) model.”
The Mechanics of n-Player Differentiable Games (2018, methodology): “We compare SGA with simultaneous gradient descent, optimistic mirror descent (Daskalakis et al., 2018) and consensus optimization (Mescheder et al.”
The Unusual Effectiveness of Averaging in GAN Training (2018, methodology): “We compare this baseline with Optimistic Adam (OMD), Consensus Optimization (CO), and Zero Centered Gradient Penalty (Zero-GP) by using the same architecture, and following the original implementations as closely as possible.”
Revisiting Stochastic Extragradient (2019, methodology): “In particular, the objective f(x, y) = x>y leads to geometrical divergence for any nontrivial initialization (Daskalakis et al., 2018).”
Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks (2018, methodology): “3.1 (Improved) Optimistic Mirror Descent Daskalakis et al. [2017] employed Optimistic Mirror Descent (OMD) motivated by online learning to solve the instability problem in GANs.”
Introduction to Multi-Armed Bandits (2019, background): “Yet, positive results are known for some special cases (Kleinberg et al., 2009b; Daskalakis et al., 2018).”
Invariant Risk Minimization Games (2020, methodology): “There are different strategies in the literature to compute the equilibrium, such as best response dynamics (BRD) and fictitious play (Fudenberg & Levine, 1998), but none of these strategies are guaranteed to arrive at equilibria in continuous games except for special classes of games (Hofbauer & So”
Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities (2021, background): “However, there is an increasing trend in applying these single-call extragradient methods to stabilize the training of generative adversarial networks (GAN) [8, 12, 34], which is nonconvex-nonconcave in general and hence has remained underexplored.”
Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games (2020, background): “On the other hand, xt can possibly converge to a limit cycle or repeatedly hit the boundary in monotone games (Mertikopoulos et al., 2018a; Daskalakis et al., 2018) despite that the time-average ( ∑t j=1 xj)/t converges.”
Global Convergence and Variance-Reduced Optimization for a Class of Nonconvex-Nonconcave Minimax Problems (2020, background): “…with constant stepsizes could fail to converge for general smooth function [Mescheder et al., 2018], even for the bilinear games [Gidel et al., 2019]; even when they do converge, the stable limit point may not be a local Nash equilibrium [Daskalakis et al., 2018, Mazumdar and Ratliﬀ, 2018].”