GAN Inversion: A Survey

By Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, Ming-Hsuan Yang. 2021

638 citations recorded by the source corpus.

GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model so that the image can be faithfully reconstructed from the inverted code by the generator. As an emerging technique to bridge the real and fake image domains, GAN inversion plays an essential role in enabling pretrained GAN models, such as StyleGAN and BigGAN, for applications of real image editing. Moreover, GAN inversion interprets GAN's latent space and examines how realistic images can be generated. In this paper, we provide a survey of GAN inversion with a focus on its representative algorithms and its applications in image restoration and image manipulation. We further discuss the trends and challenges for future research. A curated list of GAN inversion methods, datasets, and other related information can be found at https://github.com/weihaox/awesome-gan-inversion.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Generative Adversarial Networks (2021, neutral): “By providing a thorough overview of GANs, we hope to make this complex and challenging topic more accessible to researchers and practitioners in the field, and inspire new innovations and applications.”
Diffusion Autoencoders: Toward a Meaningful and Decodable Representation (2021, methodology): “Alternatively, one can use a trained GAN for extracting a representation using the so-called GAN inversion [28,58], which optimizes for a latent code that reproduces the given input.”
Designing an encoder for StyleGAN image manipulation (2021, methodology): “This task has been commonly referred to as GAN Inversion [31, 42, 45].”
Muse: Text-To-Image Generation via Masked Generative Transformers (2023, background): “Image Editing with Generative Models: GANs have been extensively studied for image editing and manipulation capabilities (see (Xia et al., 2022) for a survey).”
High-fidelity GAN Inversion with Padding Space (2022, background): “GAN inversion [45,56] aims at finding the reverse mapping of the generator in GANs.”
Pivotal Tuning for Latent-based Editing of Real Images (2021, background): “As previously mentioned, to edit a real image using latent manipulation, one must perform GAN inversion [Xia et al. 2021; Zhu et al. 2016] , meaning one must find a latent vector from which the generator would generate the input image.”
Identity-Guided Face Generation with Multi-Modal Contour Conditions (2021, methodology): “We refer readers to [12] for a comprehensive survey.”
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation (2023, background): “By properly exploiting their latent structure, prior works succeeded in image editing [33, 79, 80, 81, 82, 83, 84], semantic control [55, 56, 85], disentangled control direction discovery [86, 87, 88, 89], etc..”
Only a matter of style: age transformation using a style-based regression model (2021, background): “Recent years have seen diverse approaches for performing such an inversion [Abdal et al. 2019, 2020a; Bau et al. 2019; Creswell and Bharath 2018; Pidhorskyi et al. 2020; Richardson et al. 2020; Tewari et al. 2020; Tov et al. 2021; Xia et al. 2021; Zhu et al. 2020].”
Cycle-Consistent Inverse GAN for Text-to-Image Synthesis (2021, methodology): “As a new technique to connect the images and the GAN latent space, GAN inversion enables the pretrained GAN model to be used in various image generation applications, such as image editing [34].”
StyleAlign: Analysis and Applications of Aligned StyleGAN Models (2021, neutral): “See Xia et al. (2021) for a more thorough review.”
High-Fidelity GAN Inversion for Image Attribute Editing (2021, background): “To enable real-world image editing, GAN inversion techniques (Xia et al. 2021) have been recently explored, which aim at projecting images to the latent space of a well-trained GAN generator.”
ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation (2023, background): “GAN inversion refers to projecting real images into latent codes so that images can be faithfully reconstructed and edited with pre-trained GAN models [22, 39].”
TediGAN: Text-Guided Diverse Face Image Generation and Manipulation (2021, background): “The superior characteristics of W space appeal to numerous researchers to develop advanced GAN inversion techniques [38, 2, 1] to invert real images back into the StyleGAN’s latent space and perform meaningful manipulation.”
Towards Open-World Text-Guided Face Image Generation and Manipulation (2021, background): “The superior characteristics of W space and its variants intrigue numerous researchers to develop advanced GAN inversion techniques [11], [12] to find the inverted codes for real images in the StyleGAN’s latent space and perform meaningful manipulation afterward.”