Accelerating the Super-Resolution Convolutional Neural Network
By Chao Dong, Chen Change Loy, Xiaoou Tang. 2016
3,453 citations recorded by the source corpus.
As a successful deep model applied in image super-resolution (SR), the Super-Resolution Convolutional Neural Network (SRCNN) [1, 2] has demonstrated superior performance to the previous hand-crafted models either in speed and restoration quality. However, the high computational cost still hinders it from practical usage that demands real-time performance (24 fps). In this paper, we aim at accelerating the current SRCNN, and propose a compact hourglass-shape CNN structure for faster and better SR. We re-design the SRCNN structure mainly in three aspects. First, we introduce a deconvolution layer at the end of the network, then the mapping is learned directly from the original low-resolution image (without interpolation) to the high-resolution one. Second, we reformulate the mapping layer by shrinking the input feature dimension before mapping and expanding back afterwards. Third, we adopt smaller filter sizes but more mapping layers. The proposed model achieves a speed up of more than 40 times with even superior restoration quality. Further, we present the parameter settings that can achieve real-time performance on a generic CPU while still maintaining good performance. A corresponding transfer strategy is also proposed for fast training and testing across different upscaling factors.
Subfields: Computer Vision and Pattern Recognition, Media Technology, Efficiency & Compression
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Chen Change Loy
- Lei Zhang
- Radu Timofte
- Xiaogang Wang
- Trevor Darrell
In Media Technology
In Efficiency & Compression
Citations in the corpus
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (2016, background): “In the context of SISR it was also shown that learning upscaling filters is beneficial in terms of accuracy and speed [11, 48, 57].”
- EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis (2016, methodology): “…include dictionary-based methods [33, 40, 52, 59, 61, 64] that learn a sparse representation of image patches as a combination of dictionary atoms, as well as neural network-based approaches [4, 8, 9, 24, 25, 26, 47, 48, 62] which apply convolutional neural networks (CNNs) to the task of SISR.”
- Enhanced Deep Residual Networks for Single Image Super-Resolution (2017, methodology): “Since Dong et al. [4, 5] first proposed a deep learning-based SR method, various CNN architectures have been studied for SR. Kim et al. [11, 12] first introduced the residual network for training much deeper network architectures and achieved superior performance.”
- Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution (2017, methodology): “Ground-truth HR Bicubic FSRCNN [8] VDSR [17] LapSRN (ours) Ground-truth HR Bicubic FSRCNN [8] VDSR [17] LapSRN (ours) Figure 7: Comparison of real-world photos for 4 × SR.”
- Image Super-Resolution Using Very Deep Residual Channel Attention Networks (2018, methodology): “Manga109 (4×): YumeiroCooking HR Bicubic SRCNN [1] FSRCNN [2] VDSR [4] PSNR/SSIM 24.”
- Edge-Enhanced GAN for Remote Sensing Image Superresolution (2019, neutral): “[41] proposed Fast”
- Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network (2018, methodology): “Super-Resolution Convolutional Neural Networks (SRCNNs) [4] and their more recent Faster-SRCNNs (FSRCNNs) [5] draw a lot of attention as they showed that simple structured CNNs can produce outstanding SISR results of 2D natural images.”
- Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “Although currently there is no accepted best practice for performing or evaluating super-resolution on which channels, earlier models favor operating on the Y channel of YCbCr space [28], [45], [78], [79], while more recent models tend to operate on RGB channels [30], [33], [61], [70].”
- Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation (2017, background): “imaging [34] and natural image analysis [16], [41] because these models usually operate on local patch level.”
- Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation (2016, background): “Similar solutions to improve efficiency have also been proposed based on transposed convolutions [7, 20].”
- Lightweight Image Super-Resolution with Adaptive Weighted Learning Network (2019, methodology): “Therefore, it is still a challenging task to build the lightweight SR networks Dong et al. further proposed a faster SRCNN (FSRCNN) algorithm after SRCNN to reduce the computational cost by using the original LR image instead of the interpolated LR image as the input of the SR network [3].”
- Image De-Raining Using a Conditional Generative Adversarial Network (2017, background): “inking layers are stacked in the middle part serving for better computational efficiency. These three shrinking layers can be also regarded as performing linear combination within the learned features [42]. These are followed by a stack of deconvolutional 2 layers (along with batch normalization and ”
- Image Super-Resolution via Dual-State Recurrent Networks (2018, methodology): “[11] adopted deconvolution layers to accelerate SRCNN in combination with smaller filter sizes and more convolution layers.”