MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks

By Animesh Karnewar, Oliver Wang. 2019

224 citations recorded by the source corpus.

While Generative Adversarial Networks (GANs) have seen huge successes in image synthesis tasks, they are notoriously difficult to adapt to different datasets, in part due to instability during training and sensitivity to hyperparameters. One commonly accepted reason for this instability is that gradients passing from the discriminator to the generator become uninformative when there isn’t enough overlap in the supports of the real and fake distributions. In this work, we propose the Multi-Scale Gradient Generative Adversarial Network (MSG-GAN), a simple but effective technique for addressing this by allowing the flow of gradients from the discriminator to the generator at multiple scales. This technique provides a stable approach for high resolution image synthesis, and serves as an alternative to the commonly used progressive growing technique. We show that MSG-GAN converges stably on a variety of image datasets of different sizes, resolutions and domains, as well as different types of loss functions and architectures, all with the same set of fixed hyperparameters. When compared to state-of-the-art GANs, our approach matches or exceeds the performance in most of the cases we tried.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

Citations in the corpus

Analyzing and Improving the Image Quality of StyleGAN (2019, neutral): “Figure 7a shows MSG-GAN [22], which connects the matching resolutions of the generator and discriminator us-ing multiple skip connections.”
Projected GANs Converge Faster (2021, background): “Providing multi-scale feedback with one or multiple discriminators has been helpful for both image synthesis [23, 24] and image-to-image translation [43, 64].”
On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, methodology): “Each FID entry (row) is computed using same set of images generated by corresponding StyleGAN2 [29] variants and MSG-GAN [25] model.”
TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up (2021, methodology): “We choose MSG-GAN [84] for comparison since it extracts multi-scale representation as well.”
Generative Adversarial Networks for Face Generation: A Survey (2022, background): “Some researchers [33, 64, 65, 77] use GANs to learn progressively to generate face images from low to high resolution.”
U-Shape Transformer for Underwater Image Enhancement (2021, methodology): “Moreover, a multi-scale gradient flow mechanism [22] is utilized to guarantee the stable training process, whilst a multi-color space loss function is constructed with the RGB, LAB and LCH color space in accordance with the human eye’s cognition.”
Perception Prioritized Training of Diffusion Models (2022, methodology): “FFHQ results reproduced from [11, 32, 38], Oxford Flower from [16], and CelebA-HQ from from [45].”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “Another approach is to include skip connections between the generator and discriminator at each resolution, allowing gradients to flow through shorter paths to each layer, providing extra information to the generator [112], [117], [228].”
Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis (2021, background): “2) Learned perceptual similarity (LPIPS) (Zhang et al., 2018) provides a perceptual distance between two images.”
Thin-Plate Spline Motion Model for Image Animation (2022, background): “Multi-scale feature fusion has proven to be effective in several tasks in computer vision, including keypoint prediction [4, 29, 33], segmentation [19, 24, 36] and image generation [8, 15, 16].”
Scaling up GANs for Text-to-Image Synthesis (2023, unknown): “We believe that our GAN upsampler can serve as a drop-in replacement for the super-resolution stage of other generative models.”
Deep Generative Models in Engineering Design: A Review (2021, background): “Manufacturability: Synthesizing manufacturable designs is an important, albeit less-explored area in DGMs. Greminger [100] proposes approaching the problem of manufacturability using an MSG-GAN architecture [146] adapted to the 3D domain.”