TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network

By Ayushman Dash, John Cristian Borges Gamboa, Sheraz Ahmed, Marcus Liwicki, Muhammad Zeshan Afzal. 2017

156 citations recorded by the source corpus.

In this work, we present the Text Conditioned Auxiliary Classifier Generative Adversarial Network, (TAC-GAN) a text to image Generative Adversarial Network (GAN) for synthesizing images from their text descriptions. Former approaches have tried to condition the generative process on the textual data; but allying it to the usage of class information, known to diversify the generated samples and improve their structural coherence, has not been explored. We trained the presented TAC-GAN model on the Oxford-102 dataset of flowers, and evaluated the discriminability of the generated images with Inception-Score, as well as their diversity using the Multi-Scale Structural Similarity Index (MS-SSIM). Our approach outperforms the state-of-the-art models, i.e., its inception score is 3.45, corresponding to a relative increase of 7.8% compared to the recently introduced StackGan. A comparison of the mean MS-SSIM scores of the training and generated samples per class shows that our approach is able to generate highly diverse images with an average MS-SSIM of 0.14 over all generated classes.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Generative Models

Citations in the corpus

DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-To-Image Synthesis (2019, background): “Similar to TAC-GAN, PPGN [16] includes a conditional network to synthesize images conditioned on a caption.”
Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis (2018, background): “Recently, approaches based on conditional Generative Adversarial Network (GAN) [7] have shown promising results in text-toimage synthesis [21, 23, 34, 6, 4].”
Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View (2019, background): “large number of interesting applications of GANs [37], such as generating examples for image datasets [36], [38], creating photographs of human faces, objects, and scenes [39]–[41], generating cartoon and Pokemon characters [42], imageto-image translation [43], [44], text-to-image translation [45]–[”
Photographic Text-to-Image Synthesis with a Hierarchically-Nested Adversarial Network (2018, methodology): “HDGAN achieves competitive results with TAC-GAN on Oxford-102.”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, background): “Similarly, Text Auxiliary Classifier GAN (TAC-GAN) [57] is a semi-supervised text-to-image translation framework.”
An Introduction to Image Synthesis with Generative Adversarial Nets (2018, methodology): “Using auxiliary classiﬁers can also help in applications such as text-to-image synthesis [34] and image-to-image translation [35]. x True/Fake y”
Multimodal Intelligence: Representation Learning, Information Fusion, and Applications (2019, methodology): “the text-conditioned auxiliary classifier GAN (TAC-GAN) introduces an additional image classification task into the discriminator [210], while the text-conditioned semantic classifier”
How Generative Adversarial Networks and Its Variants Work : An Overview of GAN (2017, methodology): “In contrast, TAC-GAN [19] is based on AC-GAN by conditioning on the text description instead of the class label by simply concatenating fixed text embedding and noise vector z.”
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, background): “Very close to this approach is text conditioned auxiliary classifier GAN (TAC-GAN) (Dash et al., 2017) which conditions images on both sentence and class information which has shown to improve their structural coherence.”
Dual Adversarial Inference for Text-to-Image Synthesis (2019, background): “Several attempts have been made to leverage additional available information, such as object location [27], class label [5, 2], attention extracted from word features [30, 24] and text regeneration [24].”
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches (2018, methodology): “Text Conditioned Auxiliary Classifier GAN (TAC-GAN) which is used for generating or synthesizing images from a text description [209].”
Adversarial Text-to-Image Synthesis: A Review (2021, methodology): “GAN-INT-CLS [16] was able to generate low-resolution 64 × 64 pixel images, while TAC-GAN [31] generated 128× 128 pixel images.”
Generative Adversarial Networks (GANs) (2020, background): “Generation of realistic images has wide range of practical applications, such as anime character generation [159]–[164], image synthesis [165]–[168], super resolution [10], [124], [169]–[177], image editing and blending [178], [179], inpainting [125], [180]–[182], interactive image generation [183],”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “[90] learn text-to-image generation by redescription and text conditioned auxiliary classifier GAN (TAC-GAN) [390] is also proposed for text to image.”
Adversarial Learning of Semantic Relevance in Text to Image Synthesis (2018, methodology): “As used in other approaches (Reed et al. 2016b; Dash et al. 2017; Zhang et al. 2016; Zhang, Xie, and Yang 2018), choose any image outside the class of the reference image as a random negative.”