Diffusion-GAN: Training GANs with Diffusion

By Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, Mingyuan Zhou. International Conference on Learning Representations, 2022

343 citations recorded by the source corpus.

Generative adversarial networks (GANs) are challenging to train stably, and a promising remedy of injecting instance noise into the discriminator input has not been very effective in practice. In this paper, we propose Diffusion-GAN, a novel GAN framework that leverages a forward diffusion chain to generate Gaussian-mixture distributed instance noise. Diffusion-GAN consists of three components, including an adaptive diffusion process, a diffusion timestep-dependent discriminator, and a generator. Both the observed and generated data are diffused by the same adaptive diffusion process. At each diffusion timestep, there is a different noise-to-data ratio and the timestep-dependent discriminator learns to distinguish the diffused real data from the diffused generated data. The generator learns from the discriminator's feedback by backpropagating through the forward diffusion chain, whose length is adaptively adjusted to balance the noise and data levels. We theoretically show that the discriminator's timestep-dependent strategy gives consistent and helpful guidance to the generator, enabling it to match the true data distribution. We demonstrate the advantages of Diffusion-GAN over strong GAN baselines on various datasets, showing that it can produce more realistic images with higher stability and data efficiency than state-of-the-art GANs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9c3ceae3cf605f934cc5f04a44feae23b5252faa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models (2022, unknown): “In addition, combining DPMs with GANs (Xiao et al., 2022; Wang et al., 2022) improves the sample quality of GANs and sampling speed of DPMs.”
Improved Distribution Matching Distillation for Fast Image Synthesis (2024, methodology): “Inspired by DiffusionGAN [28,29], UFOGen [25] introduces noise injection prior to the real vs. fake classification in the discriminator, to smooth out the distributions, which stabilizes the training dynamics.”
Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, methodology): “As such, each denoising step is modeled by a conditional GAN, allowing larger step size.”
UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models (2023, background): “However, sampling from a pre-trained DPM usually requires multiple model evaluations to gradually perform denoising from Gaussian noise [13], consuming more inference time and computational costs compared with single-step generative models like GANs.”
One-Step Diffusion with Distribution Matching Distillation (2023, methodology): “Our method is different from recent works that combine GANs with diffusion [68, 81–83], as our formulation is not grounded in GANs.”
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation (2023, methodology): “Inference Time FID-5k CLIP SD 1.4-DPM Solver (25 step)[5, 19] 0”