GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution

By Kelvin C. K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy. Computer Vision and Pattern Recognition, 2020

302 citations recorded by the source corpus.

We show that pre-trained Generative Adversarial Networks (GANs), e.g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR). While most existing SR approaches attempt to generate realistic textures through learning with adversarial loss, our method, Generative LatEnt bANk (GLEAN), goes beyond existing practices by directly leveraging rich and diverse priors encapsulated in a pre-trained GAN. But unlike prevalent GAN inversion methods that require expensive image-specific optimization at runtime, our approach only needs a single forward pass to generate the upscaled image. GLEAN can be easily incorporated in a simple encoder-bank-decoder architecture with multi-resolution skip connections. Switching the bank allows the method to deal with images from diverse categories, e.g., cat, building, human face, and car. Images upscaled by GLEAN show clear improvements in terms of fidelity and texture faithfulness in comparison to existing methods as shown in Fig. 1.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/18bf61eb27c146f8e11ea0590c9b6787d8274ce6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Open-World Text-Guided Face Image Generation and Manipulation (2021, methodology): “Some recent methods [69], [70], [71] of image restoration found that inversion-based methods often generate images with low-fidelity.”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, unknown): “However, these methods fall short, either due to insufﬁcient prior expressiveness [37,75,82] or inaccurate feature matching [8], resulting in output quality that remains less than satisfactory.”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, background): “Recent works also incorporate reinforcement learning or GAN prior to image restoration [51, 6, 45].”
RePaint: Inpainting using Denoising Diffusion Probabilistic Models (2022, background): “However, similar to super-resolution methods [5, 25] that leverage the StyleGAN latent space, it is to limited specific scenarios like faces.”
ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, background): “Various SR technologies were explored from different perspectives, including network architecture [31, 32, 33, 34], image prior [35, 36, 37, 38], deep unfolding [39, 40, 41], degradation model [17, 18, 42, 43].”
Towards Robust Blind Face Restoration with Codebook Lookup Transformer (2022, unknown): “GFP-GAN [38], and GPEN [44] introduce obvious artifacts and GLEAN [3] produces over-smoothed results that lack facial details.”
Implicit Diffusion Models for Continuous Super-Resolution (2023, unknown): “Finally, similar to GLEAN [4], we train and test our model on the LSUN [47] dataset.”
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, background): “Representative GAN-prior-based meth-ods [4, 18, 55, 65] have demonstrated their capability in achieving both high-quality and high-fidelity face reconstruction.”
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “Various approaches employ generative priors, including GAN inversion [2, 4, 27, 55, 60], GAN encoders [9, 100], or using GAN as the core module for IR [77, 84].”