End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks

By Lijun Wang, Hongyu Wang, P. Li. 2016

149 citations recorded by the source corpus.

In this paper, we propose a new image super-resolution (SR) approach based on a convolutional neural network (CNN), which jointly learns the feature extraction, upsampling, and high-resolution (HR) reconstruction modules, yielding a completely end-to-end trainable deep CNN. However, directly training such a deep network in an end-to-end fashion is challenging, which takes a longer time to converge and may lead to sub-optimal results. To address this issue, we propose to jointly train an ensemble of deep and shallow networks. The shallow network with weaker learning capability restores the main structure of the image content, while the deep network with stronger representation power captures the high-frequency details. Since the shallow network is much easier to optimize, it significantly lowers the difficulty of deep network optimization during joint training. To further ensure more accurate restoration of HR images, the high-frequency details are reconstructed in a multi-scale manner to simultaneously incorporate both short- and long-range contextual information. The proposed method is extensively evaluated on widely adopted data sets and compares favorably against state-of-the-art methods. In-depth ablation studies are conducted to verify the contributions of different network designs to image SR, providing additional insights for future research.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (2016, background): “to learn the upscaling filters directly can further increase performance both in terms of accuracy and speed [40, 48].”
MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques (2023, background): “It should be noted that the training of deeper networks requires extra time than the training of shallower or simpler networks [35].”
Super-resolution reconstruction of MR image with a novel residual learning network algorithm (2018, methodology): “On the other hand, the multi-scale strategy of the conventional CNN-based SR algorithms, which are mainly implemented on small size LR images (Wang et al 2016, Dong et al 2016, Lai et al 2017, Zhao et al 2017), may not be suitable for the ILR image-based GRL in CNN.”
A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening (2017, methodology): “To make adequate use of the rich spatial information in high-resolution imagery and improve the robustness of the feature extraction among various and complex ground scenes, we introduce the multiscale convolutional layer block, which was applied to image superresolution in [30] and classification i”