Transformer for Single Image Super-Resolution

By Zhisheng Lu, Juncheng Li, Hong Liu, Chao Huang, Linlin Zhang, Tieyong Zeng. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021

531 citations recorded by the source corpus.

Single image super-resolution (SISR) has witnessed great strides with the development of deep learning. However, most existing studies focus on building more complex networks with a massive number of layers. Recently, more and more researchers start to explore the application of Transformer in computer vision tasks. However, the heavy computational cost and high GPU memory occupation of the vision Transformer cannot be ignored. In this paper, we propose a novel Efficient Super-Resolution Transformer (ESRT) for SISR. ESRT is a hybrid model, which consists of a Lightweight CNN Backbone (LCB) and a Lightweight Transformer Backbone (LTB). Among them, LCB can dynamically adjust the size of the feature map to extract deep features with a low computational costs. LTB is composed of a series of Efficient Transformers (ET), which occupies a small GPU memory occupation, thanks to the specially designed Efficient Multi-Head Attention (EMHA). Extensive experiments show that ESRT achieves competitive results with low computational cost. Compared with the original Transformer which occupies 16,057M GPU memory, ESRT only occupies 4,191M GPU memory. All codes are available at https://github.com/luissen/ESRT.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/916d10920b79cef09b14e86810637ab3c70e5956

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Qiangqiang Yuan
2. Liangpei Zhang

In Computer Vision and Pattern Recognition

1. Liangpei Zhang
2. Qiangqiang Yuan
3. Zijie Li
4. Jiang He
5. Chia-Wen Lin
6. Yi Xiao
7. Kui Jiang
8. Dule Shu

Citations in the corpus

• TTST: A Top-k Token Selective Transformer for Remote Sensing Image Super-Resolution (2024, methodology): “To further assess the effectiveness of our MFL, we conduct a comparison with the standard MLP layer, which is widely used in Transformers [27], [30], [43].”
• A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction (2022, background): “…research progress in deep learning for image super-resolution , such as the progressive GAN training approach, 36–38 the transformer-based approach, 39–41 and the diffusion model-based approach, 42–44 several neural network models have been proposed to increase the resolution and reconstruct…”