Adversarial Text-to-Image Synthesis: A Review

By Stanislav Frolov, Tobias Hinz, Federico Raue, Jörn Hees, Andreas R. Dengel. 2021

207 citations recorded by the source corpus.

With the advent of generative adversarial networks, synthesizing images from text descriptions has recently become an active research area. It is a flexible and intuitive way for conditional image generation with significant progress in the last years regarding visual realism, diversity, and semantic alignment. However, the field still faces several challenges that require further research efforts such as enabling the generation of high-resolution images with multiple objects, and developing suitable and reliable evaluation metrics that correlate with human judgement. In this review, we contextualize the state of the art of adversarial text-to-image synthesis models, their development since their inception five years ago, and propose a taxonomy based on the level of supervision. We critically examine current strategies to evaluate text-to-image synthesis models, highlight shortcomings, and identify new areas of research, ranging from the development of better datasets and evaluation metrics to possible improvements in architectural design and model training. This review complements previous surveys on generative adversarial networks with a focus on text-to-image synthesis which we believe will help researchers to further advance the field.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, background): “In recent years, focus has been on methods that aim to learn a direct text-to-image mapping function, often in the form of a conditional GAN (Goodfellow et al., 2014a; Mirza & Osindero, 2014; Reed et al., 2016; Frolov et al., 2021).”
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers (2022, background): “4 SOA only focuses on the existence of objects, which makes it not well suited to evaluate object attributes and relation between objects [20, 27].”
Text-to-image Diffusion Models in Generative AI: A Survey (2023, background): “The other stream of surveys [21], [23], [24] focuses on textto-image task but is limited to the GAN-based approaches, making them somewhat outdated considering the recent trend of the diffusion model replacing GAN.”
TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering (2023, background): “However, they rely on ground-truth images and are based on a classiﬁcation model, which makes them not suitable for complex datasets [12].”
DALL-EVAL: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models (2022, background): “Generating images from textual descriptions based on machine learning is an active research area [21].”