Seven Ways to Improve Example-Based Single Image Super Resolution

By Radu Timofte, Rasmus Rothe, Luc Van Gool. Computer Vision and Pattern Recognition, 2015

449 citations recorded by the source corpus.

In this paper we present seven techniques that everybody should know to improve example-based single image super resolution (SR): 1) augmentation of data, 2) use of large dictionaries with efficient search structures, 3) cascading, 4) image self-similarities, 5) back projection refinement, 6) enhanced prediction by consistency check, and 7) context reasoning. We validate our seven techniques on standard SR benchmarks (i.e. Set5, Set14, B100) and methods (i.e. A+, SRCNN, ANR, Zeyde, Yang) and achieve substantial improvements. The techniques are widely applicable and require no changes or only minor adjustments of the SR methods. Moreover, our Improved A+ (IA) method sets new stateof-the-art results outperforming A+ by up to 0.9dB on average PSNR whilst maintaining a low time complexity.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/af0892ab043b1a37f3e8f81b148c66369d64d391

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Image Super Resolution Based on Fusing Multiple Convolution Neural Networks (2017, methodology): “The typical methods include neighbor embedding [7], Markov network [22], kernel regression [4], random forest [26], sparse coding [25][9][10][12], and convolution neural networks.”
NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study (2017, methodology): “They propose a diversity of ideas and design details and generally build upon and go beyond the very recent proposed SR works [10, 49, 21, 26].”
NTIRE 2018 Challenge on Single Image Super-Resolution: Methods and Results (2018, methodology): “Train data Data augmentation by scaling (only Track 1), flipping, and rotation [33] is another commonly used technique.”
Image Super-Resolution via Deep Recursive Residual Network (2017, methodology): “Shallow (non-DL) models include A+ [31], SelfEx [10], RFL [23], NBSRF [22], PSyCo [20] and IA [30].”
Balanced Two-Stage Residual Networks for Image Super-Resolution (2017, methodology): “Existing methods can be broadly classified into three categories: interpolation based [5], image statistics based [6, 32] and example based methods [42, 34, 36].”
Multiscale brain MRI super-resolution using deep 3D convolutional networks (2019, methodology): “Following Timofte et al. (2016), who have experimentally shown several ways to improve SR techniques from a baseline ar-94 chitecture, we study the impact of eight key elements on the performance of convolutional neural networks for 3D brain MRI SR.”
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks (2018, methodology): “e interpolation. used for learning the perceptual index, is also employed in our training; 3) a high weight of loss L 1 up to = 10 is used due to the PSNR constraints; 4) we also use back projection [46] as post-processing, which can improve PSNR and sometimes lower the perceptual index. For other r”
Enhanced Deep Residual Networks for Single Image Super-Resolution (2017, methodology): “In order to maximize the potential performance of our model, we adopt the self-ensemble strategy similarly to [28].”
Deep Back-Projection Networks for Super-Resolution (2018, background): “[45] mentioned that backprojection can improve the quality of SR image.”
Residual Feature Distillation Network for Lightweight Image Super-Resolution (2020, background): “Image SR is a very active research area where many approaches [25,15] have been proposed to generate the upscaled images.”
Zero-Shot Super-Resolution Using Deep Internal Learning (2017, background): “For the sake of robustness, as well as to allow large SR scale factors s even from very small LR images, the SR is performed gradually [5, 21].”
Image Super-Resolution Using Dense Skip Connections (2017, background): “Such dictionary-based methods [25] have achieved comparable state-of-the-art results.”
MADNet: A Fast and Lightweight Network for Single-Image Super Resolution (2020, methodology): “The geometric self-ensembling strategy [27], [41] is used for further evaluation and marked with “ + ” in this article.”
EDVR: Video Restoration With Enhanced Deformable Convolutional Networks (2019, methodology): “In the competition, we adopt selfensemble as [42, 20].”
NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results (2017, methodology): “Train data DIV2K dataset [1] has 800 train images and all the competitors found the amount of data sufficient for training their model, especially after data augmentation (by operations such as flipping, rotation, scaling [34]).”
Residual Dense Network for Image Super-Resolution (2018, unknown): “Similar to [29, 16], we also adopt self-ensemble strategy [16] to further improve our RDN and denote the self-ensembled RDN as RDN+.”
NTIRE 2019 Challenge on Video Deblurring and Super-Resolution: Dataset and Study (2019, methodology): “Still, as seen in the single image super-resolution literature [30, 8, 32, 1, 31, 2, 4] for further progress in video deblurring and SR, standardized benchmarks are essential to allow comparison of different proposed methods under the same conditions.”