Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

By Liangbin Xie, Chao Dong, Ying Shan. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021

2,005 citations recorded by the source corpus.

Though many attempts have been made in blind super-resolution to restore low-resolution images with unknown and complex degradations, they are still far from addressing general real-world degraded images. In this work, we extend the powerful ESRGAN to a practical restoration application (namely, Real-ESRGAN), which is trained with pure synthetic data. Specifically, a high-order degradation modeling process is introduced to better simulate complex real-world degradations. We also consider the common ringing and overshoot artifacts in the synthesis process. In addition, we employ a U-Net discriminator with spectral normalization to increase discriminator capability and stabilize the training dynamics. Extensive comparisons have shown its superior visual performance than prior works on various real datasets. We also provide efficient implementations to synthesize training pairs on the fly.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/cb75d3f6c08fea6338afd147cab92531e514c5cd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

MAXIM: Multi-Axis MLP for Image Processing (2022, background): “Creating more realistic, large-scale datasets through data-generation schemes [80, 92] can improve this shortcoming.”
eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers (2022, methodology): “To train the super-resolution models, we condition on the ground-truth low-resolution inputs that are corrupted by random degradation [81].”
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment (2021, methodology): “We keep the architecture unchanged and adopt the secondorder degradation model [32] during training.”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, positive): “We adopt the degradation pipeline of Real-ESRGAN [63] to synthesize LR/HR pairs on DIV2K [1], DIV8K [22], Flickr2K [58] and OutdoorSceneTraining [64] datasets.”
KNN-Diffusion: Image Generation via Large-Scale Retrieval (2022, methodology): “To further up-sample the images to 512 × 512 and 1024 × 1024 we used the open-source super resolution provided by (Wang et al., 2021).”
MambaIR: A Simple Baseline for Image Restoration with State-Space Model (2024, methodology): “6: Qualitative comparison with RealSR [32], ESRGAN [68], BSRGAN [80], Real-ESRGAN [67], and SwinIR [41] on real image super-resolution with scale × 4.”
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, neutral): “(3) We build another real-world dataset, named RealLR200 , which comprises 38 LR images used in recent literature [34, 57, 69], 47 LR images from DiffBIR [40], 50 LR images from Vide-oLQ (the last frame of each video sequence) [4], and 65 LR images collected from the internet by ourselves.”
ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “Various SR technologies were explored from different perspectives, including network architecture [31, 32, 33, 34], image prior [35, 36, 37, 38], deep unfolding [39, 40, 41], degradation model [17, 18, 42, 43].”
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, methodology): “Regarding high-order degradation, we follow [55] to use the second-order degradation, which repeats the classical degradation model: blur-resize-noise process”
SinSR: Diffusion-Based Image Super-Resolution in a Single Step (2023, methodology): “…sample ϵ ∼ N ( 0 , κ 2 η T I ) 5: x T = y + ϵ 6: for t = T, T − 1 , ..., 1 do else 10: 14: ˆ x T = f ˆ θ ( x 0 , y, 0) , 16: Perform a gradient descent step on ∇ ˆ θ L 19: end while 20: return The student model f ˆ θ . with ResShift [46] where the degradation model is adopted from RealESRGAN [41].”
Scaling up GANs for Text-to-Image Synthesis (2023, neutral): “We also use the unconditional Real-ESRGAN [26] as another baseline.”
SwinIR: Image Restoration Using Swin Transformer (2021, methodology): “For real-world image SR, we use a combination of pixel loss, GAN loss and perceptual loss [81, 89, 80, 27, 39, 81] to improve visual quality.”
Activating More Pixels in Image Super-Resolution Transformer (2022, background): “To improve perceptual quality, [24,53,64,52] introduce adversarial learning to generate more realistic results.”
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “Low-Quality Input Real-ESRGAN+[78] StableSR[75] DiffBIR[49] PASD[85] SUPIR (ours) SUPIR (ours).”