To learn image super-resolution, use a GAN to learn how to do image degradation first

By Adrian Bulat, Jing Yang, Georgios Tzimiropoulos. European Conference on Computer Vision, 2018

402 citations recorded by the source corpus.

This paper is on image and face super-resolution. The vast majority of prior work for this problem focus on how to increase the resolution of low-resolution images which are artificially generated by simple bilinear down-sampling (or in a few cases by blurring followed by down-sampling). We show that such methods fail to produce good results when applied to real-world low-resolution, low quality images. To circumvent this problem, we propose a two-stage process which firstly trains a High-to-Low Generative Adversarial Network (GAN) to learn how to degrade and downsample high-resolution images requiring, during training, only unpaired high and low-resolution images. Once this is achieved, the output of this network is used to train a Low-to-High GAN for image super-resolution using this time paired low- and high-resolution images. Our main result is that this network can be now used to effectively increase the quality of real-world low-resolution images. We have applied the proposed pipeline for the problem of face super-resolution where we report large improvement over baselines and prior work although the proposed method is potentially applicable to other object categories.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1b599545cec03dcc2c4174a25978f1ebd0d43715

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “Bulat et al. [128] propose a two-stage process which ﬁrstly trains an HR-to-LR GAN to learn degradation using unpaired LR-HR images and then trains an LR-to-HR GAN for image super-resolution using paired LR-HR images conducted base on the ﬁrst GAN.”
Unsupervised Learning for Real-World Super-Resolution (2019, background): “[5] focus on the problem of learning the downsampling process.”
Unsupervised Image Super-Resolution with an Indirect Supervised Path (2019, methodology): “The model [2] is mainly supervised with an adversarial loss and a pixel-wise loss is used to speed up convergence.”
Video Enhancement with Task-Oriented Flow (2017, background): “2017; Niklaus and Liu 2018), super-resolution (Huang et al. 2015; Kappeler et al. 2016; Tao et al. 2017; Bulat et al. 2018; Ahn et al. 2018; Jo et al. 2018), denoising Mildenhall et al.”
Blind Super-Resolution With Iterative Kernel Correction (2019, background): “Since the seminal work of employing convolutional neural networks (CNNs) for SR [6], various deep learning based methods with different network architectures [15, 16, 18, 29, 41, 10, 40] and training strategies [19, 34, 27, 5] have been proposed to continuously improve the SR performance.”
Deblurring by Realistic Blurring (2020, methodology): “The idea of data augmentation has been widely applied in different fields [31], like face verification [20] and SR [1].”
Frequency Separation for Real-World Super-Resolution (2019, methodology): “[3] explore a method to learn the downsampling operation.”
PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models (2020, background): “However, methods have shown that the degradations can be estimated in entirely unsupervised fashions for arbitrary LR images (that is, not necessarily those which have been downscaled bicubically) [6, 25].”
Real-World Single Image Super-Resolution: A Brief Review (2021, methodology): “com/ sefibk/KernelGAN 20The source code of FISR [95] is available at https://github.”
Creating High Resolution Images with a Latent Adversarial Generator (2020, background): “While the choice of a down-scaling operator is an interesting topic of its own [3], our main goal is generate images and evaluate the quality of our output’s robustness to lack of such information.”
A Deep Journey into Super-resolution (2019, background): “One recent effort towards the solution of this problem ﬁrst learns a GAN to model the real-world degradation [108].”
NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results (2020, background): “This scenario has recently attracted significant interest due to its high relevance in applications [74, 37, 8, 45].”
Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network (2020, background): “Impressive works [61,71] exist in literature to create realistic LR images from HR images.”