CPGAN: Full-Spectrum Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis

By Jiadong Liang, Wenjie Pei, Feng Lu. 2019

19 citations recorded by the source corpus.

Typical methods for text-to-image synthesis seek to design effective generative architecture to model the text-to-image mapping directly. It is fairly arduous due to the cross-modality translation. In this paper we circumvent this problem by focusing on parsing the content of both the input text and the synthesized image thoroughly to model the text-to-image consistency in the semantic level. Particularly, we design a memory structure to parse the textual content by exploring semantic correspondence between each word in the vocabulary to its various visual contexts across relevant images during text encoding. Meanwhile, the synthesized image is parsed to learn its semantics in an object-aware manner. Moreover, we customize a conditional discriminator to model the fine-grained correlations between words and image sub-regions to push for the text-image semantic alignment. Extensive experiments on COCO dataset manifest that our model advances the state-of-the-art performance significantly (from 35.69 to 52.73 in Inception Score).

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP (2022, background): “Early *Corresponding author. ar X iv :2 20 3. approaches (e.g., DMGAN [45], AttnGAN [39], DF-GAN [35], Obj-GAN [19], OPGAN [19], SD-GAN [40], CPGAN [21], XMC-GAN [41]) that directly generate pixels from the given text embeddings with a convolutional generator have shown promising revolutions to gene”
Cross-Modal Contrastive Learning for Text-to-Image Generation (2021, methodology): “Although CP-GAN achieves higher IS and SOA scores, both our human evaluations and visual inspection of randomly selected images indicates XMC-GAN’s image quality is much higher than CP-GAN’s.”
TediGAN: Text-Guided Diverse Image Generation and Manipulation (2020, unknown): “The previous methods usually use a submodule named Deep At-tentional Multimodal Similarity Model (DAMSM) [52, 26, 27, 29, 45] as the text-image matching loss, which learns two neural networks that map subregions of the image and words of the sentence to a common semantic space, thus measures the…”