Unsupervised Learning for Real-World Super-Resolution

By Andreas Lugmayr, Martin Danelljan, Radu Timofte. 2019

182 citations recorded by the source corpus.

Most current super-resolution methods rely on low and high resolution image pairs to train a network in a fully supervised manner. However, such image pairs are not available in real-world applications. Instead of directly addressing this problem, most works employ the popular bicubic downsampling strategy to artificially generate a corresponding low resolution image. Unfortunately, this strategy introduces significant artifacts, removing natural sensor noise and other real-world characteristics. Super-resolution networks trained on such bicubic images therefore struggle to generalize to natural images. In this work, we propose an unsupervised approach for image super-resolution. Given only unpaired data, we learn to invert the effects of bicubic downsampling in order to restore the natural image characteristics present in the data. This allows us to generate realistic image pairs, faithfully reflecting the distribution of real-world images. Our super-resolution network can therefore be trained with direct pixel-wise supervision in the high resolution domain, while robustly generalizing to real input. We demonstrate the effectiveness of our approach in quantitative and qualitative experiments.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, background): “[26] propose to learn a deep degradation mapping by employing a cycle consistency loss and then generate LR/HR pairs for supervised training.”
Unsupervised Image Super-Resolution with an Indirect Supervised Path (2019, methodology): “[21] also proposed to combine an unsupervised image translation model and an image restoration model, but they are trained separately.”
Real-World Super-Resolution via Kernel Estimation and Noise Injection (2020, unknown): “[24] pro-pose an unsupervised learning method for real-world super-resolution.”
Frequency Separation for Real-World Super-Resolution (2019, methodology): “The validation and test set contain 100 images each and have the same corrup-tions as the source data [26].”
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results (2020, methodology): “The setting is similar to that from the NTIRE 2019 challenge on real image super-resolution [3] employing RealSR dataset [4], and is different from the AIM 2019 [25] and NTIRE 2020 [24] challenges on real-world super-resolution where no LR-HR pairs are available for training, therefore an unsupervis”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, methodology): “The training pairs are usually 1) captured with specific cameras followed by tedious alignments [5, 49]; 2) or directly learned from unpaired data with cycle consistency loss [52, 32]; 3) or synthesized with estimated blur kernels and extracted noise patches [58, 18].”
SRFlow: Learning the Super-Resolution Space with Normalizing Flow (2020, background): “andard delity oriented metrics, Peak Signal to Noise Ratio (PSNR) and structural similarity index (SSIM) [48], although they are known to not correlate well with the human perception of image quality [17,23,27,29,43,45]. Furthermore, we report the noreference metrics NIQE [33], BRISQUE [32] and PIQU”
Real-World Single Image Super-Resolution: A Brief Review (2021, methodology): “ULSR ICCVW-2019 [109] Domain translation-based Unsupervised learning, Unpaired data, GAN”
NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results (2020, methodology): “For Track 2, the team adopts the CycleSR framework [24, 45] to generate degrade images.”
Deep Unfolding Network for Image Super-Resolution (2020, background): “While learning-based blind image restoration has recently received considerable attention [12, 39, 43, 50, 62], we note that this work focuses on non-blind SISR which assumes the LR image, blur kernel and noise level are known beforehand.”