GAN Inversion for Out-of-Range Images with Geometric Transformations

By Kyoungkook Kang, S. Kim, Sunghyun Cho. 2021

75 citations recorded by the source corpus.

For successful semantic editing of real images, it is critical for a GAN inversion method to find an in-domain latent code that aligns with the domain of a pre-trained GAN model. Unfortunately, such in-domain latent codes can be found only for in-range images that align with the training images of a GAN model. In this paper, we propose BDInvert, a novel GAN inversion approach to semantic editing of out- of-range images that are geometrically unaligned with the training images of a GAN model. To find a latent code that is semantically editable, BDInvert inverts an input out-of-range image into an alternative latent space than the original latent space. We also propose a regularized inversion method to find a solution that supports semantic editing in the alternative space. Our experiments show that BDInvert effectively supports semantic editing of out-of-range images with geometric transformations.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

1. Daniel Cohen-Or
2. Or Patashnik
3. Yuval Alaluf
4. Eli Shechtman
5. Yujun Shen
6. Dani Lischinski
7. Zongze Wu
8. Rinon Gal

In Computer Vision and Pattern Recognition

1. Daniel Cohen-Or
2. Ming-Hsuan Yang
3. Luc Van Gool
4. Eli Shechtman
5. Chen Change Loy
6. Andrea Vedaldi
7. Alexei A. Efros
8. Dacheng Tao

In Generative Models

1. Daniel Cohen-Or
2. Eli Shechtman
3. Dani Lischinski
4. Or Patashnik
5. Yuval Alaluf
6. Rinon Gal
7. Amit H. Bermano
8. Yujiu Yang

Citations in the corpus

• GAN Inversion: A Survey (2021, methodology): “[63] propose a method to invert out-ofrange images.”
• HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, methodology): “Generally, inversion methods either directly optimize the latent vector to minimize the error for a given image [1, 2, 7, 14, 21, 40, 70, 74, 75], train an encoder over a large number of samples to learn a mapping from an image to its latent representation [5,22,22,30, 36, 43, 49, 50, 53, 61, 65], o”
• High-Fidelity GAN Inversion for Image Attribute Editing (2021, methodology): “Existing GAN inversion approaches either perform perimage optimization [1, 17, 45] or learn a data-driven encoder [26, 34].”
• High-fidelity GAN Inversion with Padding Space (2022, methodology): “With the startof-the-art synthesis performance, they are widely used for GAN inversion studies [1,2,21,35,36,40,41,45,51,54].”
• Third Time's the Charm? Image and Video Editing with StyleGAN3 (2022, methodology): “Inversion methods typically directly optimize the latent vector to minimize the reconstruction error of a given image [1, 2, 10, 17, 24, 43, 77, 81, 86], train an encoder to map an image to its latent representation [6, 25, 31, 38, 45, 49, 51, 55, 67, 71], or design a hybrid approach combining both.”
• HyperInverter: Improving StyleGAN Inversion via Hypernetwork (2021, methodology): “There are two general approaches to determining a latent code of a GAN model given an input image: by an iterative optimization [1, 9, 20, 25, 33] and by inference with an encoder [36, 38, 45, 58].”