Towards Real-World Blind Face Restoration with Generative Facial Prior

By Xintao Wang, Yu Li, Honglun Zhang, Ying Shan. Computer Vision and Pattern Recognition, 2021

644 citations recorded by the source corpus.

Blind face restoration usually relies on facial priors, such as facial geometry prior or reference prior, to restore realistic and faithful details. However, very low-quality inputs cannot offer accurate geometric prior while high-quality references are inaccessible, limiting the applicability in real-world scenarios. In this work, we propose GFP-GAN that leverages rich and diverse priors encapsulated in a pretrained face GAN for blind face restoration. This Generative Facial Prior (GFP) is incorporated into the face restoration process via spatial feature transform layers, which allow our method to achieve a good balance of realness and fidelity. Thanks to the powerful generative facial prior and delicate designs, our GFP-GAN could jointly restore facial details and enhance colors with just a single forward pass, while GAN inversion methods require image-specific optimization at inference. Extensive experiments show that our method achieves superior performance to prior art on both synthetic and real-world datasets.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/54483c5672bbee68cb77969d9cf6681209230b95

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

The Creativity of Text-to-Image Generation (2022, methodology): “through a model that was trained on improving faces (such as GFPGAN [60]), and then proceed to perform more advanced techniques (e.”
Implicit Diffusion Models for Continuous Super-Resolution (2023, positive): “Unlike GAN-based methods that rely on additional priors [4, 44], our conditioning network only provides encoded multi-resolution features.”
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “Various approaches employ generative priors, including GAN inversion [2, 4, 27, 55, 60], GAN encoders [9, 100], or using GAN as the core module for IR [77, 84].”
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, methodology): “For BFR task, we evaluate our method on a synthetic dataset CelebA-Test [39] and three real-world datasets: LFW-Test [54], CelebChild-Test [54], and WIDER-Test [68].”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, neutral): “However, designing these constraints assumes knowing the image degradations a priori, which is typically unknown and complex.”
Towards Open-World Text-Guided Face Image Generation and Manipulation (2021, methodology): “Some recent methods [69], [70], [71] of image restoration found that inversion-based methods often generate images with low-fidelity.”
Pretraining is All You Need for Image-to-Image Translation (2022, background): “There are prior attempts [37, 9, 56, 49, 51, 33, 31] to leverage a pretrained model as generative prior for conditional image synthesis, image editing and restoration.”
Towards Robust Blind Face Restoration with Codebook Lookup Transformer (2022, neutral): “GFP-GAN [38], and GPEN [44] introduce obvious artifacts and GLEAN [3] produces over-smoothed results that lack facial details.”
RODIN: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion (2022, methodology): “Thus, at this stage, we jointly train a convolution refiner [67] on our data which complements the missing details of the NeRF rendering, ultimately producing compelling 1024 × 1024 image outputs.”
High-Fidelity GAN Inversion for Image Attribute Editing (2021, background): “This Unet-like structure is adopted by some recent image restoration works [6,37] that conveys latent maps (e.”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, background): “Recent works also incorporate reinforcement learning or GAN prior to image restoration [51, 6, 45].”