Conditional Image Generation and Manipulation for User-Specified Content

By David Stap, Maurits J. R. Bleeker, Sarah Ibrahimi, Maartje ter Hoeve. 2020

36 citations recorded by the source corpus.

In recent years, Generative Adversarial Networks (GANs) have improved steadily towards generating increasingly impressive real-world images. It is useful to steer the image generation process for purposes such as content creation. This can be done by conditioning the model on additional information. However, when conditioning on additional information, there still exists a large set of images that agree with a particular conditioning. This makes it unlikely that the generated image is exactly as envisioned by a user, which is problematic for practical content creation scenarios such as generating facial composites or stock photos. To solve this problem, we propose a single pipeline for text-to-image generation and manipulation. In the first part of our pipeline we introduce textStyleGAN, a model that is conditioned on text. In the second part of our pipeline we make use of the pre-trained weights of textStyleGAN to perform semantic facial image manipulation. The approach works by finding semantic directions in latent space. We show that this method can be used to manipulate facial images for a wide range of attributes. Finally, we introduce the CelebTD-HQ dataset, an extension to CelebA-HQ, consisting of faces and corresponding textual descriptions.

Subfields: Computer Vision and Pattern Recognition, Law

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yujiu Yang
2. Weihao Xia
3. Jing-Hao Xue
4. Baoyuan Wu
5. Daniel Cohen-Or
6. Dani Lischinski

Citations in the corpus

• Towards Open-World Text-Guided Face Image Generation and Manipulation (2021, background): “A similar dataset [55] is also proposed but it is not publicly available.”
• TediGAN: Text-Guided Diverse Image Generation and Manipulation (2020, background): “A similar dataset [43] is also proposed but it is not publicly available.”