Designing a Practical Degradation Model for Deep Blind Image Super-Resolution

By Kai Zhang, Jingyun Liang, Luc Van Gool, Radu Timofte. 2021

963 citations recorded by the source corpus.

It is widely acknowledged that single image super-resolution (SISR) methods would not perform well if the assumed degradation model deviates from those in real images. Although several degradation models take additional factors into consideration, such as blur, they are still not effective enough to cover the diverse degradations of real images. To address this issue, this paper proposes to design a more complex but practical degradation model that consists of randomly shuffled blur, downsampling and noise degradations. Specifically, the blur is approximated by two convolutions with isotropic and anisotropic Gaussian kernels; the downsampling is randomly chosen from nearest, bilinear and bicubic interpolations; the noise is synthesized by adding Gaussian noise with different noise levels, adopting JPEG compression with different quality factors, and generating processed camera sensor noise via reverse-forward camera image signal processing (ISP) pipeline model and RAW image noise model. To verify the effectiveness of the new degradation model, we have trained a deep blind ES-RGAN super-resolver and then applied it to super-resolve both synthetic and real images with diverse degradations. The experimental results demonstrate that the new degradation model can help to significantly improve the practicability of deep super-resolvers, thus providing a powerful alternative solution for real SISR applications.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Hierarchical Text-Conditional Image Generation with CLIP Latents (2022, methodology): “For the first upsampling stage, we use gaussian blur [43], and for the second, we use a more diverse BSR degradation [42, 59].”
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, methodology): “According to the recent BSR survey [37], the most popular solutions may be BSRGAN [64] and Real-ESRGAN [55].”
MambaIR: A Simple Baseline for Image Restoration with State-Space Model (2024, methodology): “6: Qualitative comparison with RealSR [32], ESRGAN [68], BSRGAN [80], Real-ESRGAN [67], and SwinIR [41] on real image super-resolution with scale × 4.”
Flow-based Kernel Prior with Application to Blind Super-Resolution (2021, background): “Obviously, these two methods do not make full use of the anisotropic Gaussian kernel prior which has been demonstrated to be effective enough for real image SR [11, 40, 50, 54, 55].”
SinSR: Diffusion-Based Image Super-Resolution in a Single Step (2023, methodology): “We compare our method with several representative SR models, including RealSR-JPEG [12], ESRGAN [40], BSRGAN [47], SwinIR [18], RealESR-GAN [ 41], DASR [19], LDM [32], and ResShift [46].”
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “Our method can handle a wide range of degradations, and we compare it with the latest methods with the same capabilities, including BSRGAN [90], Real-ESRGAN [78], StableSR [75], DiffBIR [49] and PASD [85].”
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, neutral): “Datasets Metrics BSRGAN [70] Real-[57] ESRGAN LDL [34] DASR [35] FeMaSR [5] LDM [46] StableSR [55] ResShift [66] PASD [65]”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, positive): “Speciﬁcally, BSRGAN [77] and Real-ESRGAN [63] present effective degradation pipelines for blind SR in real world.”
ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “Various SR technologies were explored from different perspectives, including network architecture [31, 32, 33, 34], image prior [35, 36, 37, 38], deep unfolding [39, 40, 41], degradation model [17, 18, 42, 43].”
SwinIR: Image Restoration Using Swin Transformer (2021, methodology): “In addition, to exploit the full potential of SwinIR for real 1https://github.com/cszn/BSRGAN Table 2: Quantitative comparison (average PSNR/SSIM) with state-of-the-art methods for classical image SR on bench- mark datasets.”
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, methodology): “The first category is GAN-based methods, including BSRGAN (Zhang et al. 2021), Real-ESRGAN (Wang et al. 2021b), SwinIR-GAN (Liang et al. 2021), LDL (Liang, Zeng, and Zhang 2022) and FeMaSR (Chen et al. 2022).”
Towards Flexible Blind JPEG Artifacts Removal (2021, methodology): “Note that our double JPEG degradation model can also be applied to other tasks such as blind single image super-resolution [49].”
Pretraining is All You Need for Image-to-Image Translation (2022, methodology): “Following [20, 40], we apply random degradation, speciﬁcally the real-world BSR degradation [58], on the training inputs to reduce the gap between training images and the samples from the base model.”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, methodology): “A recent work [53] also proposes a random shuffling strategy to synthesize more practical degradations.”