KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis

By Hongchen Tan, Xiuping Liu, Meng Liu, Baocai Yin, Xin Li. 2020

74 citations recorded by the source corpus.

This paper presents a new framework, Knowledge-Transfer Generative Adversarial Network (KT-GAN), for fine-grained text-to-image generation. We introduce two novel mechanisms: an Alternate Attention-Transfer Mechanism (AATM) and a Semantic Distillation Mechanism (SDM), to help generator better bridge the cross-domain gap between text and image. The AATM updates word attention weights and attention weights of image sub-regions alternately, to progressively highlight important word information and enrich details of synthesized images. The SDM uses the image encoder trained in the Image-to-Image task to guide training of the text encoder in the Text-to-Image task, for generating better text features and higher-quality images. With extensive experimental validation on two public datasets, our KT-GAN outperforms the baseline method significantly, and also achieves the competive results over different evaluation metrics.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

  1. Dacheng Tao

In Computer Vision and Pattern Recognition

  1. Dacheng Tao
  2. Jianping Gou
  3. S. Maybank
  4. Baosheng Yu
  5. Luc Van Gool
  6. Trevor Darrell
  7. Xiaogang Wang
  8. Ming-Hsuan Yang

In Generative Models

  1. Dacheng Tao

In Multimodal Learning

  1. Dacheng Tao
  2. S. Maybank
  3. Baosheng Yu
  4. Jianping Gou

Citations in the corpus