SwinIR: Image Restoration Using Swin Transformer
By Jingyun Liang, Jie Cao, Guolei Sun, Kai Zhang, Luc Van Gool, Radu Timofte. 2021
4,894 citations recorded by the source corpus.
Image restoration is a long-standing low-level vision problem that aims to restore high-quality images from low-quality images (e.g., downscaled, noisy and compressed images). While state-of-the-art image restoration methods are based on convolutional neural networks, few attempts have been made with Transformers which show impressive performance on high-level vision tasks. In this paper, we propose a strong baseline model SwinIR for image restoration based on the Swin Transformer. SwinIR consists of three parts: shallow feature extraction, deep feature extraction and high-quality image reconstruction. In particular, the deep feature extraction module is composed of several residual Swin Transformer blocks (RSTB), each of which has several Swin Transformer layers together with a residual connection. We conduct experiments on three representative tasks: image super-resolution (including classical, lightweight and real-world image super-resolution), image denoising (including grayscale and color image denoising) and JPEG compression artifact reduction. Experimental results demonstrate that SwinIR outperforms state-of-the-art methods on different tasks by up to 0.14∼0.45dB, while the total number of parameters can be reduced by up to 67%.
Subfields: Computer Vision and Pattern Recognition, Media Technology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Chen Change Loy
- Xiaogang Wang
- Lei Zhang
- Trevor Darrell
- Yu Qiao
In Media Technology
Citations in the corpus
- StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets (2022, methodology): “the data with a super-resolution network [Liang et al. 2021], see supplementary.”
- Transformers in Vision: A Survey (2021, methodology): “After witnessing the success of Transformer models in high-level vision problems, numerous Transformer-based methods have been proposed for low-level vision tasks, including image super-resolution [30, 125, 228], denoising [30, 217], deraining [30, 217], and colorization [109].”
- Dilated Neighborhood Attention Transformer (2022, background): “It also inspired other works that extended it to different tasks beyond the ones explored in the paper, such as generation [48], restoration [22], masked image modeling [45], video action recognition [28], and more.”
- DiffIR: Efficient Diffusion Model for Image Restoration (2023, methodology): “Presently, the transformer has been adopted in numerous vision tasks, such as image recognition [17, 60], segmentation [62, 69, 87, 49], object detection [5, 89], and image restoration [7, 38, 74, 36, 8].”
- Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (2022, background): “…(ViTs) [35, 61, 89, 98], which have shown leading performances on many visual tasks – not only image classification [35, 109] and representation learning [4, 10, 17, 105], but also many downstream tasks such as object detection [25,61], semantic segmentation [98, 103] and image restoration [11, 56].”
- Efficient Long-Range Attention Network for Image Super-resolution (2022, methodology): “Very recently, transformer has also been applied to low-level vision tasks [28, 55, 58, 9].”
- SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer (2022, methodology): “Thus, following the literature [33], [38], we alternately utilize regular and shifted window partitioning to enable cross-window connections, where shifted window partitioning means shifting the feature by (bM2 c, b M 2 c) pixels before partitioning.”
- MAXIM: Multi-Axis MLP for Image Processing (2022, methodology): “Unlike previous low-level Transformers [9, 14, 51, 95], MAXIM has several desired properties, making it intriguing for image processing tasks.”
- ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “We evaluate the effectiveness of ResShift in comparison to seven recent SR methods, namely ESRGAN [61], RealSR-JPEG [62], BSRGAN [17], RealESRGAN [18], SwinIR [19], DASR [20], and LDM [11].”
- Uformer: A General U-Shaped Transformer for Image Restoration (2021, background): “For example, squeeze-and-excitation networks [20] and non-local neural networks [60] inspire a branch of methods for different image restoration tasks, such as super-resolution [35, 79], deraining [32,74], and denoising [73,74].”
- Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, background): “Over time, the need for blind IR methods that are not based on specific degradation assumptions has grown [5, 10, 25, 34, 35, 46–48 , 91].”
- U-Shape Transformer for Underwater Image Enhancement (2021, methodology): “So in the second round, we invited volunteers to vote on each reference picture again to select its existing problems and determine the corresponding optimization method, and then use appropriate image enhancement methods [56,33,55] to process it.”
- Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring (2022, background): “…model the global contexts and achieves significant progress in lots of high-level vision tasks (e.g., image classification [11], object detection [1,31] and semantic segmentation [25,32]), it has been developed to solve image super-resolution [10], image deblurring [21,27] and image denoise [2,24].”
- Simple Baselines for Image Restoration (2022, background): “Alternatively, [21,20,34] apply self-attention only in a fix-sized local window to alleviate the issue of increased computation.”
- MambaIR: A Simple Baseline for Image Restoration with State-Space Model (2024, methodology): “In accordance with previous works [10, 41, 78], we perform data augmentation by applying horizontal flips and random rotations of 90 [42].”
- MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment (2022, background): “First, the convolutional layer with spatially invariant filters can enhance the translational equivariance [18].”
- Image Restoration with Mean-Reverting Stochastic Differential Equations (2023, background): “Various CNN-based architectures have been proposed (Za-mir et al., 2021; Chen et al., 2022), and recently the use of transformers has also been extensively explored (Liang et al., 2021; Zamir et al., 2022; Luo et al., 2022b).”