ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
By Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Chen Change Loy, Yu Qiao, Xiaoou Tang. 2018
4,766 citations recorded by the source corpus.
The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGAN – network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture recovery. Benefiting from these improvements, the proposed ESRGAN achieves consistently better visual quality with more realistic and natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge (region 3) with the best perceptual index. The code is available at https://github.com/xinntao/ESRGAN.
Subfields: Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Chen Change Loy
- Xiaogang Wang
- Yi Yang
- Trevor Darrell
- Lei Zhang
In Generative Models
- Daniel Cohen-Or
- Eli Shechtman
- Richard Zhang
- Jun-Yan Zhu
- Chen Change Loy
- Tero Karras
- Bolei Zhou
- Yujun Shen
Citations in the corpus
- Image Super-Resolution via Iterative Refinement (2021, methodology): “We compare SR3 with EnhanceNet [31], ESRGAN [30], SRFlow [32], FSRGAN [14] and PULSE [17].”
- Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, methodology): “Although significant improvements on the PSNR [27, 61] and perceptual quality [24, 49] have been achieved, such methods usually do not perform well on real images.”
- Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, background): “To address the problem that the texture information generated by SRGAN is not real enough, and is often accompanied by some noise, Wang et al. [45] proposed an Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN).”
- AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, positive): “These encoder-decoder GANs also gain popularity in image enhancement (Wang et al., 2018a; Kupyn et al., 2019; Jiang et al., 2019) and editing (Sanakoyeu et al., 2018; Yang et al., 2019).”
- The Perception-Distortion Tradeoff (2017, background): “GAN-based image restoration Most recently, GAN-based methods have demonstrated unprecedented perceptual quality in super-resolution [3], [6], [9], [37], inpainting [7], [20], [38], compression [21], [39], [40], deblurring [41] and image-to-image translation [11], [22], [42].”
- StarGAN v2: Diverse Image Synthesis for Multiple Domains (2019, background): “Generative adversarial networks (GANs) [9] have shown impressive results in many computer vision tasks such as image synthesis [3, 25, 7], colorization [15, 36] and superresolution [21, 34].”
- PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models (2020, methodology): “To provide another measure of perceptual quality, we evaluated the Naturalness Image Quality Evaluator (NIQE) score [16], previously used in perceptual super-resolution [10, 4, 24].”
- Towards Real-World Blind Face Restoration with Generative Facial Prior (2021, methodology): “We also compare our GFP-GAN with image restoration methods: RCAN [75], ESRGAN [66] and DeblurGANv2 [41], and we finetune them on our face training set for fair comparisons.”
- Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution (2020, methodology): “Following [37], we train our models on DIV2K [34] and Flickr2K [26] datasets.”
- Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “[77] feature similarity MS-SSIM [74] multi-scale SSIM IFC [75] information fidelity criterion PSNR peak signal-to-noise ratio IQA image quality assessment SSIM [62] structural similarity MOS mean opinion score VIF [76] visual information fidelity MSSIM [62] mean SSIM BN [140] batch normalization GAN [”
- Deblurring by Realistic Blurring (2020, background): “Recently, deep learning methods have achieved a great success in the areas of object recognition [9, 44, 43, 19, 18] and image reconstruction including video deblurring [45], video dehazing [25], and other GAN-based generation tasks [29, 24, 39, 37].”
- GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution (2020, background): “It is noteworthy that the performance of DGP and ESRGAN+ are less promising on categories other than human faces.”
- Activating More Pixels in Image Super-Resolution Transformer (2022, methodology): “To improve perceptual quality, [25, 55, 56, 67] introduce adversarial learning to generate more realistic results.”
- Residual Feature Aggregation Network for Image Super-Resolution (2020, background): “3(Left) depicts a basic residual module used in EDSR [18] and ESRGAN [31].”
- ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “We evaluate the effectiveness of ResShift in comparison to seven recent SR methods, namely ESRGAN [61], RealSR-JPEG [62], BSRGAN [17], RealESRGAN [18], SwinIR [19], DASR [20], and LDM [11].”
- Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains (2020, methodology): “…which has been generated by transfer learning, and the original "base model" have a close relationship [4] and it has been shown that linearly interpolating between the weights of two such generative models produces outputs which are approximately an interpolation of the two learned domains [11].”