Precise Recovery of Latent Vectors from Generative Adversarial Networks

By Zachary Chase Lipton, Subarna Tripathi. 2017

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

GAN Inversion: A Survey (2021, methodology): “Optimization-based GAN inversion typically optimizes the latent code based on either gradient descent [24], [25], [80], [80], [84], [113], [113] or other iterative algorithms [55], [114].”
Designing an encoder for StyleGAN image manipulation (2021, methodology): “Generally, inversion methods either (i) directly optimize the latent vector to minimize the error for the given image [1, 2, 8, 21, 23], (ii) train an encoder to map the given image to the latent space [13, 28, 31], or (iii) use a hybrid approach combining both [4, 45].”
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation (2020, methodology): “Generally, inversion methods either directly optimize the latent vector to minimize the error for the given image [25, 8, 1, 2], train an encoder to map the given image to the latent space [31, 8, 32, 12, 29], or use a hybrid approach combining both [4, 42].”
Paint by Word (2021, background): “Because several powerful editing methods are enabled by GAN latent manipulation, the problem of inverting a GAN is of widespread interest and is the subject of an active line of ongoing research [53, 26, 6, 1, 2, 14, 36, 52, 55, 16].”
Seeing What a GAN Cannot Generate (2019, background): “Later work found that DCGAN left-inverses can be computed to high precision [25, 46], and that inversions of a GAN for glyphs can reveal speciﬁc strokes that the generator is unable to generate [9].”
ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement (2021, background): “Compared to per-image latent vector optimization [28, 10, 1, 2, 25], encoders are significantly faster, as they invert using a single forward pass, and converge to areas of the latent space which are more suitable for editing [50, 40].”
Pros and Cons of GAN Evaluation Measures (2018, background): “, [12, 35, 58, 51]) assess the quality of internal representations by checking whether they satisfy certain properties, such as being “disentangled”.”
Generative Adversarial Networks: An Overview (2017, background): “Several techniques have been proposed to invert the generator of pre-trained GANs [17], [18].”
Image Processing Using Multi-Code GAN Prior (2019, methodology): “One is to directly optimize the latent code by minimizing the reconstruction error through back-propagation [30, 12, 32].”
Transforming and Projecting Images into Class-conditional Generative Networks (2020, methodology): “enerative models. The common techniques include: (1) Optimization-based methods: they nd the latent vector that can closely reconstruct the input image using gradient-based method (e.g., ADAM, LBFGS) [65,9,41,61,54,52,10] or MCMC [20], (2) Encoder-based methods: they learn an encoder to directly pre”
InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs (2020, background): “To solve this problem, a common practice is to get the reverse mapping from the image space to the latent space, which is also known as GAN Inversion [8], [42], [43], [44], [45].”
Adversarial Cross-Modal Retrieval (2017, unknown): “…our approach was inspired by the effectiveness of adversarial learning for various applications, like learning discriminative image features [17], or (un)supervised domain adaptation to enforce domain-invariant features [2, 6, 41], and regularizing correlation loss between cross-modal items…”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, methodology): “Optmization-based methods iteratively optimize a reconstruction objective [99, 92, 56, 1, 36, 16, 75, 74] or perform Markov chain Monte Carlo [21], while encoder-”
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, methodology): “Generally, inversion methods either directly optimize the latent vector to minimize the error for a given image [1, 2, 7, 14, 21, 40, 70, 74, 75], train an encoder over a large number of samples to learn a mapping from an image to its latent representation [5,22,22,30, 36, 43, 49, 50, 53, 61, 65], o”
High-fidelity GAN Inversion with Padding Space (2022, background): “Active attempts broadly fall into two categories, which are optimizationbased [7,11,17,29,32,35] and learning-based [2,33,34,36,40, 45, 51–53].”
DragDiffusion: Harnessing Diffusion Models for Interactive Point-Based Image Editing (2023, background): “Recently, due to the impressive generation results from large-scale text-to-image diffusion models [38, 42], many diffusion-based image editing methods have been proposed [4, 6, 7, 15, 22, 26, 29, 30, 32, 34, 50].”
LIA: Latently Invertible Autoencoder with Adversarial Learning (2019, methodology): “The algorithms we compare are Glow that is the state-of-the-art algorithm in normalizing ﬂows [19], the MSE-based optimization methods [30, 2, 25] 2 , the adversarially learned inference (ALI) [8], and the adversarial generator-encoder (AGE) network [40].”