Deep Networks for Image Super-Resolution with Sparse Prior

By Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, Thomas S. Huang. 2015

812 citations recorded by the source corpus.

Deep learning techniques have been successfully applied in many areas of computer vision, including low-level image restoration problems. For image super-resolution, several models based on deep neural networks have been recently proposed and attained superior performance that overshadows all previous handcrafted models. The question then arises whether large-capacity and data-driven models have become the dominant solution to the ill-posed super-resolution problem. In this paper, we argue that domain expertise represented by the conventional sparse coding model is still valuable, and it can be combined with the key ingredients of deep learning to achieve further improved results. We show that a sparse coding model particularly designed for super-resolution can be incarnated as a neural network, and trained in a cascaded structure from end to end. The interpretation of the network based on sparse coding leads to much more efficient and effective training, as well as a reduced model size. Our model is evaluated on a wide range of images, and shows clear advantage over existing state-of-the-art methods in terms of both restoration accuracy and human subjective quality.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Image Super-Resolution via Iterative Refinement (2021, neutral): “sion based and trained with an MSE loss [49], [52], [54], [55], [56].”
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (2018, methodology): “We also evaluate several additional methods, including bicubic upsampling, and four of the top performing deep superresolution methods [24, 58, 31, 47].”
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (2016, background): “[50] the authors encode a sparse representation prior into their feed-forward network architecture based on the learned iterative shrinkage and thresholding algorithm (LISTA) [19].”
Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation (2018, background): “Modern approaches to super-resolution are built on deep convolutional networks [5, 45, 27] as well as generative adversarial networks [18, 13] which use an adversarial loss to imagine high-resolution details in RGB images.”
Image Super-Resolution Using Very Deep Residual Channel Attention Networks (2018, methodology): “Urban100 (8×): img 040 HR Bicubic SRCNN [1] SCN [3] VDSR [4] PSNR/SSIM 15.”
Image Super-Resolution via Deep Recursive Residual Network (2017, methodology): “The deep models (d ≤ 8) include SRCNN [2], DJSR [33], CSCN [32], ESPCN [25] and FSRCNN [3].”
NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results (2017, methodology): “The common approach is the enhanced prediction or multi-view processing [34, 36] which assumes flips and rotations (in 90◦ steps) of the input LR image to obtain 4 or 8 HR results that are”
Finding Tiny Faces in the Wild with Generative Adversarial Network (2018, background): “With the development of deep learning, great improvements have been achieved on super-resolution [5, 6, 15, 26].”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, background): “Other networks designed to perform SR include recursive learning [62], [63], [64], progressive reconstruction [65], [66], dense connections [7], [67], [68], attention mechanisms [69], [70], [71], multi-branch learning [66], [72], [73], [74], and generative adversarial networks (GANs) [68], [75], [76”
Image Super-Resolution by Neural Texture Transfer (2019, methodology): “[37] combined the strengths of sparse coding and deep network and made considerable improvement over previous models.”
Seven Ways to Improve Example-Based Single Image Super Resolution (2015, neutral): “[35].”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, background): “Other networks designed to perform SR include recursive learning [57,41,3], progressive reconstruction [105,60], dense connections [99,104,127], attention mechanisms [125,23,126], multi-branch learning [60,66,22,64], and generative adversarial networks (GANs) [104,76,87,63].”
Pyramid-Structured Depth MAP Super-Resolution Based on Deep Dense-Residual Network (2019, neutral): “AR [22], JID [23], SCN [24], MS [10], Lap [13] and MFR [11].”
URetinex-Net: Retinex-based Deep Unfolding Network for Low-light Image Enhancement (2022, methodology): “Inspired by such optimization scheme, deep unfolding algorithms have made great impact on many significant image processing problems, such as super-resolution [33] [36], image denoising [4] [38], clutter suppression [31], and rain removal [8].”
End-to-End Kernel Learning with Supervised Convolutional Kernel Networks (2016, methodology): “Lemma 1 (Perturbation view of backpropagration.)”
Depth Map Super-Resolution by Deep Multi-Scale Guidance (2016, methodology): “In distinction to other single-image super resolution CNNs [11,12,22], our networks do not use a single fixed (non-trainable) upsampling operator.”