Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model

By Jianrui Cai, Huiyu Zeng, Hongwei Yong, Zisheng Cao, Lei Zhang. 2019

752 citations recorded by the source corpus.

Most of the existing learning-based single image super-resolution (SISR) methods are trained and evaluated on simulated datasets, where the low-resolution (LR) images are generated by applying a simple and uniform degradation (i.e., bicubic downsampling) to their high-resolution (HR) counterparts. However, the degradations in real-world LR images are far more complicated. As a consequence, the SISR models trained on simulated data become less effective when applied to practical scenarios. In this paper, we build a real-world super-resolution (RealSR) dataset where paired LR-HR images on the same scene are captured by adjusting the focal length of a digital camera. An image registration algorithm is developed to progressively align the image pairs at different resolutions. Considering that the degradation kernels are naturally non-uniform in our dataset, we present a Laplacian pyramid based kernel prediction network (LP-KPN), which efficiently learns per-pixel kernels to recover the HR image. Our extensive experiments demonstrate that SISR models trained on our RealSR dataset deliver better visual quality with sharper edges and finer textures on real-world scenes than those trained on simulated datasets. Though our RealSR dataset is built by using only two cameras (Canon 5D3 and Nikon D810), the trained model generalizes well to other camera devices such as Sony a7II and mobile phones.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

In Evaluation & Benchmarks

Citations in the corpus

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, neutral): “[7] and Wei et al .”
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, neutral): “To this end, some researchers proposed to collect real-world LR-HR image pairs using long-short camera focal lens [3, 61].”
ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “We evaluate the effectiveness of ResShift in comparison to seven recent SR methods, namely ESRGAN [62], RealSR-JPEG [63], BSRGAN [18], RealESRGAN [19], SwinIR [20], DASR [21], and LDM [11].”
DiffIR: Efficient Diffusion Model for Image Restoration (2023, background): “From level-1 to level-4, the attention heads in DMTA are [1, 2, 4, 8], and the number of channels is [48, 96, 192, 384].”
Evaluation and development of deep neural networks for image super-resolution in optical microscopy (2021, methodology): “4e and Supplementary Video 7), which was presumably propelled by hydrodynamic forces generated by cytoplasmic streaming 24 .”
SinSR: Diffusion-Based Image Super-Resolution in a Single Step (2023, methodology): “RealSR [3] and Re-alSet65 [46] are adopted to evaluate the generalization ability of the model on unseen real-world data.”
Unsupervised Image Super-Resolution with an Indirect Supervised Path (2019, background): “[3, 33] take a further step to capture LR-HR image pairs under realistic setting, that is, tuning focal length of DSLR cameras to collect images of different resolution.”
Real-World Super-Resolution via Kernel Estimation and Noise Injection (2020, background): “[40, 5, 4, 45] collect raw photo pairs directly from nature scene with particular camera equipment.”
Component Divide-and-Conquer for Real-World Image Super-Resolution (2020, methodology): “We compare our method with several state-of-the-art SR methods, including SRResNet [13], EDSR [14], ESRGAN [26], RCAN [33], LP-KPN [2] and DDet [22].”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, methodology): “We compare our Real-ESRGAN with several stateof-the-art methods, including ESRGAN [48], DAN [33], CDC [49], RealSR [18] and BSRGAN [53].”
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results (2020, methodology): “[3] released another real image SR dataset, named RealSR, which was captured from two DSLR cameras.”
Transformer for Single Image Super-Resolution (2021, methodology): “In addition, ESRT achieves better performance than LK-KPN on × 4 , which was speciﬁcally designed for the real SR task.”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, methodology): “LP-KPN [14]”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, neutral): “Hence, we set the coefﬁcient w to 0.5 by default for trading between quality and ﬁdelity.”
Real-World Single Image Super-Resolution: A Brief Review (2021, methodology): “[81] is similar to RealSR [80], with a larger scale.”
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, methodology): “(Cai et al. 2019) collected a real-world dataset with paired LQ-HQ images by zooming camera lens.”
Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms (2020, methodology): “To resolve this, we perform photometric alignment based on a linear model following [3, 34, 1].”
Blind Super-Resolution Kernel Estimation using an Internal-GAN (2019, methodology): “A different benchmark suggested by [4] is not yet available, and is only restricted to 3 SR blur kernels.”