HyperInverter: Improving StyleGAN Inversion via Hypernetwork

By Tan M. Dinh, Anh Tuan Tran, Rang Nguyen, Binh-Son Hua. 2021

129 citations recorded by the source corpus.

Real-world image manipulation has achieved fantastic progress in recent years as a result of the exploration and utilization of GAN latent spaces. GAN inversion is the first step in this pipeline, which aims to map the real image to the latent code faithfully. Unfortunately, the majority of existing GAN inversion methods fail to meet at least one of the three requirements listed below: high reconstruction quality, editability, and fast inference. We present a novel two-phase strategy in this research that fits all requirements at the same time. In the first phase, we train an encoder to map the input image to StyleGAN2 W-space, which was proven to have excellent editability but lower reconstruction quality. In the second phase, we supplement the reconstruction ability in the initial phase by leveraging a series of hypernetworks to recover the missing information during inversion. These two steps complement each other to yield high reconstruction quality thanks to the hypernetwork branch and excellent editability due to the inversion done in the W-space. Our method is entirely encoder-based, resulting in extremely fast inference. Extensive experiments on two challenging datasets demonstrate the superiority of our method.11Project page: https://di-mi-ta.github.io/HyperInverter

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Adding Conditional Control to Text-to-Image Diffusion Models (2023, background): “Successful results of HyperNetwork are also reported in image generation using generative adversarial networks [1, 10] and other machine learning tasks [51].”
Third Time's the Charm? Image and Video Editing with StyleGAN3 (2022, background): “While most techniques keep the generator ﬁxed, recent works have proposed tuning the generator to achieve more accurate image inversions, either via a per-image optimization [10,56] or a learned hypernet-work [7,19].”
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion (2022, background): “There it is typical to apply a delicate tuning of a generative model to better reconstruct speciﬁc faces or scenes (Bau et al., 2019; Roich et al., 2021; Alaluf et al., 2021; Dinh et al., 2022; Cao et al., 2022; Nitzan et al., 2022).”
High-fidelity GAN Inversion with Padding Space (2022, background): “Some also [3,10,38] try to fine-tune the pre-trained generator, leading to better inversion but much slower speed.”
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, background): “[17] also explore the use of hypernetworks for achieving higher fidelity inversions.”