StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

By Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris N. Metaxas. 2016

2,951 citations recorded by the source corpus.

Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing textto- image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) to generate 256.256 photo-realistic images conditioned on text descriptions. We decompose the hard problem into more manageable sub-problems through a sketch-refinement process. The Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. It is able to rectify defects in Stage-I results and add compelling details with the refinement process. To improve the diversity of the synthesized images and stabilize the training of the conditional-GAN, we introduce a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold. Extensive experiments and comparisons with state-of-the-arts on benchmark datasets demonstrate that the proposed method achieves significant improvements on generating photo-realistic images conditioned on text descriptions.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

  1. Jun-Yan Zhu
  2. Eli Shechtman
  3. Richard Zhang
  4. Alexei A. Efros
  5. Tianfu Wu
  6. Björn Ommer
  7. Robin Rombach
  8. Andrea Vedaldi

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Yi Yang
  8. Bernt Schiele

In Generative Models

  1. Jun-Yan Zhu
  2. Richard Zhang
  3. Eli Shechtman
  4. Tianfu Wu
  5. Alexei A. Efros
  6. Stefano Ermon
  7. Daniel Cohen-Or
  8. Bolei Zhou

Citations in the corpus