On the Limitations of First-Order Approximation in GAN Dynamics

60 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

A Closer Look at the Optimization Landscapes of Generative Adversarial Networks (2019, methodology): “Another extension was proposed by (Li et al., 2018a) where they use contour plots to look at the 2D loss surface deﬁned by two directions chosen appropriately.”
Reducing Noise in GAN Training with Variance Reduced Extragradient (2019, methodology): “Using the same formulation as the one in Mescheder et al. (2017) and Gidel et al.”
ContraGAN: Contrastive Learning for Conditional Image Generation (2020, background): “The studies on non-convexity of objective landscapes [10, 11, 12] and gradient vanishing problems [3, 11, 13, 14] emphasize the instability of the adversarial dynamics.”
Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting (2018, background): “We hypothesise that this bottleneck would likely occur in the GAN as currently GANs are still relatively unstable to train as they suffer from vanishing gradients and mode collapse [39], however we leave these research questions for future work.”
Competitive Gradient Descent (2019, background): “From this point of view it is not surprising that the convergent modifications of GDA are, implicitly or explicitly, based on higher order approximations (see also (Li et al., 2017)).”