DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis

By Ming Tao, Hao Tang, Fei Wu, Xiaoyuan Jing, Bingkun Bao, Changsheng Xu. Computer Vision and Pattern Recognition, 2020

308 citations recorded by the source corpus.

Synthesizing high-quality realistic images from text descriptions is a challenging task. Existing text-to-image Generative Adversarial Networks generally employ a stacked architecture as the backbone yet still remain three flaws. First, the stacked architecture introduces the entanglements between generators of different image scales. Second, existing studies prefer to apply and fix extra networks in adversarial learning for text-image semantic consistency, which limits the supervision capability of these networks. Third, the cross-modal attention-based text-image fusion that widely adopted by previous works is limited on several special image scales because of the computational cost. To these ends, we propose a simpler but more effective Deep Fusion Generative Adversarial Networks (DF-GAN). To be specific, we propose: (i) a novel one-stage text-to-image backbone that directly synthesizes high-resolution images without entanglements between different generators, (ii) a novel Target-Aware Discriminator composed of Matching-Aware Gradient Penalty and One-Way Output, which enhances the text-image semantic consistency without introducing extra networks, (iii) a novel deep text-image fusion block, which deepens the fusion process to make a full fusion between text and visual features. Compared with current state-of-the-art methods, our proposed DF-GAN is simpler but more efficient to synthesize realistic and text-matching images and achieves better performance on widely used datasets. Code is available at https://github.com/tobran/DF-GAN.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/10f86e4979a0d625b1e0fc6a1a45613b8977d440

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis (2023, methodology): “The stacked MLP-based conditioning layers in DF-GAN (Tao et al., 2022) implicitly include similar 2 nd order terms.”
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation (2023, background): “Later, the field shifted its attention to more complex scenarios, particularly in the MS COCO dataset [48], leading to advancements in training and generation [49, 50, 51].”
Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis (2022, methodology): “Text-to-Image Synthesis There are mainly three types of models for text-to-image synthesis: GANbased (Tao et al., 2022; Zhu et al., 2019; Li et al., 2019; Fu et al., 2020; El-Nouby et al., 2019), autoregressive (Gu et al., 2022a; Lee et al., 2022; Ding et al., 2022) and diffusion models (Liu et…”
Scaling up GANs for Text-to-Image Synthesis (2023, unknown): “We also adapt several techniques commonly used in the diffusion context and confirm that they bring similar benefits to GANs.”
GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models (2023, unknown): “Compared to earlier models based on generative adversarial networks (GANs) or variational autoencoders (VAE) [63,86, 107], more recent models [66,67] leveraging diffusion models produce signiﬁcantly higher quality images.”
Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models (2023, background): “Early works studied text-guided image synthesis in the context of GANs [Tao et al. 2022; Xu et al. 2018; Ye et al. 2021; Zhang et al. 2021; Zhu et al. 2019].”
Training-Free Layout Control with Cross-Attention Guidance (2023, background): “For several years, generative adversarial networks (GANs) [10] have been the dominant approach in image generation from textual prompts [30, 46, 47, 38, 40, 45].”
MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing (2023, background): “Early image generation methods conditioned on text description mainly based on GANs [26, 43, 44, 39, 14, 2, 46, 40, 42, 34], due to their powerful capability of high-fidelity image synthesis.”
Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis (2022, background): “…under standard T2I setting AttnGAN (Xu et al. 2018) 33.10 23.61 - Obj-GAN (Li et al. 2019) 36.52 24.09 - DM-GAN (Zhu et al. 2019) 27.34 32.32 - DF-GAN (Tao et al. 2022) 21 To summarize, we highlight that our PyU-Net framework equips DM with the ability to learn in a coarse-to-ﬁne fashion with a…”