Feedback Network for Image Super-Resolution

By Z. Li, Jinglei Yang, Zheng Liu, Xiaomin Yang, Gwanggil Jeon, Wei Wu. Computer Vision and Pattern Recognition, 2019

821 citations recorded by the source corpus.

Recent advances in image super-resolution (SR) explored the power of deep learning to achieve a better reconstruction performance. However, the feedback mechanism, which commonly exists in human visual system, has not been fully exploited in existing deep learning based image SR methods. In this paper, we propose an image super-resolution feedback network (SRFBN) to refine low-level representations with high-level information. Specifically, we use hidden states in a recurrent neural network (RNN) with constraints to achieve such feedback manner. A feedback block is designed to handle the feedback connections and to generate powerful high-level representations. The proposed SRFBN comes with a strong early reconstruction ability and can create the final high-resolution image step by step. In addition, we introduce a curriculum learning strategy to make the network well suitable for more complicated tasks, where the low-resolution images are corrupted by multiple types of degradation. Extensive experimental results demonstrate the superiority of the proposed SRFBN in comparison with the state-of-the-art methods. Code is avaliable at https://github.com/Paper99/SRFBN_CVPR19.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/96c05f7f2c2435a7793f17c2ee72cd824e26441a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Ming-Hsuan Yang
3. Lei Zhang
4. Chen Change Loy
5. Dacheng Tao
6. Radu Timofte
7. Ling Shao
8. Yi Yang

In Media Technology

1. Radu Timofte
2. Kai Zhang
3. Luc Van Gool
4. Chao Dong
5. Lei Zhang
6. Wangmeng Zuo
7. Yulun Zhang
8. Jinjin Gu

Citations in the corpus

• UIEC^2-Net: CNN-based Underwater Image Enhancement Using Two Color Space (2021, background): “Unlike other low-level vision tasks such as image super-resolution [35], image denoising [36], image deblurring [37] and image de-hazing [38], which can use plenty of synthetic degraded images and high-quality counterparts for training, it is difficult to synthesize realistic underwater images for…”
• Efficient Transformer for Single Image Super-Resolution (2021, methodology): “For example, SRFBN [1] uses a feedback manner to share the weights of several middle layers and enhance the reconstruction ability of the model iteratively.”
• Interactive Knowledge Distillation (2020, background): “Over the past few years, deeper and deeper convolutional neural networks (CNNs) have demonstrated cutting edge performance on various computer vision tasks [7,11,17,16].”
• Plug-and-Play Image Restoration With Deep Denoiser Prior (2020, methodology): “SRFBN is a recurrent neural network with feed-back mechanism.”
• A Deep Journey into Super-resolution (2019, background): “[55] proposed a Super-resolution Feedback Network (SRFBN) based on a recurrent architecture design.”
• Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution (2020, background): “Many efforts have been made to improve the performance of SR, including the interpolation-based approaches [19] and reconstructionbased methods [16, 25, 51].”
• MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation (2022, methodology): “Before training, we use 2× or 3× super-resolution [17] to first upsample the training datasets with lower resolution to a higher one (at least 1000p).”
• Real-World Super-Resolution via Kernel Estimation and Noise Injection (2020, background): “Super-Resolution Recently, many Convolutional Neural Networks (CNN)-based SR networks [23, 30, 13, 14, 22, 31, 41] achieve strong performance on bicubic downsampling images.”
• Structure-Preserving Super Resolution With Gradient Guidance (2020, background): “[28] propose a feedback framework to refine the super-resolved results step by step.”
• Lightweight Image Super-Resolution with Information Multi-distillation Network (2019, background): “[17] exploited feedback mechanism that enhancing low-level representation with high-level ones.”
• SwinIR: Image Restoration Using Swin Transformer (2021, background): “Since several revolutionary work [18, 40, 90, 91], convolutional neural networks (CNN) have become the primary workhorse for image restoration [43, 51, 43, 81, 92, 95, 93, 46, 89, 88].”
• Efficient Long-Range Attention Network for Image Super-resolution (2022, methodology): “We compare the visual quality of SR results by our ELAN and six representative models, including EDSR [29], RDN [62], SAN [10], RCAN [62], IGNN [27] and SwinIR [28], on × 4 upscaling task.”
• Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “In contrast, the SRFBN [86] uses this strategy for SR under adverse conditions, i.”
• Residual Feature Distillation Network for Lightweight Image Super-Resolution (2020, background): “,11,23,17,7,16,32] have been proposed and achieved prominent performance in image SR.”
• Dual Aggregation Transformer for Image Super-Resolution (2023, methodology): “We compare our models: DAT-S, DAT-2, and DAT, with state-of-the-art methods: EDSR [14], SRMDNF [27], RDN [32], OISR [9], RCAN [30], NLRN [15], RNAN [31], SRFBN [12], SAN [5], RFANet [16], IGNN [34], HAN [23], CSNLN [22], NLSA [21], CRAN [33], ELAN [29], DFSA [17], SwinIR [13], and CAT-A [4].”
• LatticeNet: Towards Lightweight Image Super-Resolution with Lattice Block (2020, background): “SRFBN [25] proposes a feedback mechanism to address the feedback connections and generate effective high-level feature representations.”
• MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation (2020, methodology): “Before training, we use 2× or 3× super-resolution [76] to first upsample the training datasets with lower resolution to a higher one (at least 1000p).”
• Residual Feature Aggregation Network for Image Super-Resolution (2020, methodology): “Following these pioneering works, many CNN-based methods have been proposed and achieved state-of-the-art results in SISR [15, 18, 40, 38, 3, 17, 7, 9, 39].”
• Meta-Transfer Learning for Zero-Shot Super-Resolution (2020, background): “Recently, numerous CNN-based networks have been proposed to super-resolve LR image with known downsampling kernel [15, 17, 21, 12, 47, 2, 36, 20, 13].”