Image Super-Resolution via Dual-State Recurrent Networks

By Wei Han, Shiyu Chang, Ding Liu, Mo Yu, Michael J. Witbrock, Thomas S. Huang. 2018

241 citations recorded by the source corpus.

Advances in image super-resolution (SR) have recently benefited significantly from rapid developments in deep neural networks. Inspired by these recent discoveries, we note that many state-of-the-art deep SR architectures can be reformulated as a single-state recurrent neural network (RNN) with finite unfoldings. In this paper, we explore new structures for SR based on this compact RNN view, leading us to a dual-state design, the Dual-State Recurrent Network (DSRN). Compared to its single-state counterparts that operate at a fixed spatial resolution, DSRN exploits both low-resolution (LR) and high-resolution (HR) signals jointly. Recurrent signals are exchanged between these states in both directions (both LR to HR and HR to LR) via delayed feedback. Extensive quantitative and qualitative evaluations on benchmark datasets and on a recent challenge demonstrate that the proposed DSRN performs favorably against state-of-the-art algorithms in terms of both memory consumption and predictive accuracy. The code for our method is publicly available1.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Denoising Prior Driven Deep Neural Network for Image Restoration (2018, methodology): “By treating deep superresolution architecture as a single-state recurrent neural network (RNN), in [34] a dual-state RNN has been proposed for SR to exploit both low-resolution and high-resolution signals jointly.”
Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “[77] feature similarity MS-SSIM [74] multi-scale SSIM IFC [75] information ﬁdelity criterion PSNR peak signal-to-noise ratio IQA image quality assessment SSIM [62] structural similarity MOS mean opinion score VIF [76] visual information ﬁdelity MSSIM [62] mean SSIM BN [140] batch normalization GAN [”
Meta-SR: A Magnification-Arbitrary Network for Super-Resolution (2019, background): “Both DBPN [9] and DSRN [8] made use of the mutual dependencies of low- and high-resolution images.”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, background): “Other networks designed to perform SR include recursive learning [57,41,3], progressive reconstruction [105,60], dense connections [99,104,127], attention mechanisms [125,23,126], multi-branch learning [60,66,22,64], and generative adversarial networks (GANs) [104,76,87,63].”
2018 PIRM Challenge on Perceptual Image Super-resolution (2018, methodology): “While the first attempt to apply a conv-net to image SR [6] did not significantly surpass the performance of prior methods, it set the ground for major improvements in PSNR/SSIM values over the course of the several following years [15, 17, 18, 19, 39, 11, 52, 34, 10, 51].”
Feedback Network for Image Super-Resolution (2019, background): “Recent studies [22, 10] have shown that many networks with recurrent structure (e.”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, background): “Other networks designed to perform SR include recursive learning [62], [63], [64], progressive reconstruction [65], [66], dense connections [7], [67], [68], attention mechanisms [69], [70], [71], multi-branch learning [66], [72], [73], [74], and generative adversarial networks (GANs) [68], [75], [76”
The 2018 PIRM Challenge on Perceptual Image Super-Resolution (2018, methodology): “While the first attempt to apply a conv-net to image SR [6] did not significantly surpass the performance of prior methods, it set the ground for major improvements in PSNR/SSIM values over the course of the several following years [15, 17, 18, 19, 39, 11, 52, 34, 10, 51].”
Non-Local Recurrent Network for Image Restoration (2018, methodology): “We do not list other methods [19, 33, 25, 34, 16] since their performances are worse than that of DRRN or MemNet.”
Image Super-Resolution by Neural Texture Transfer (2019, background): “As in other fields of computer vision studies, the introduction of convolutional neural networks (CNNs) [5, 37, 22, 25, 35, 13] has greatly advanced the state-of-the-art of SISR.”
Channel-Wise and Spatial Feature Modulation Network for Single Image Super-Resolution (2018, background): “In addition, [29], [30] incorporated the feedback mechanism into network designs for The feature-modulation memory (FMM) module in (a), which exploits a chain of channel-wise and spatial attention residual (CSAR) blocks to capture more informative features and utilizes the gated fusion (GF) node to…”