Second-Order Attention Network for Single Image Super-Resolution

By Tao Dai, Jianrui Cai, Yongbing Zhang, Shutao Xia, Lei Zhang. 2019

1,809 citations recorded by the source corpus.

Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and obtained remarkable performance. However, most of the existing CNN-based SISR methods mainly focus on wider or deeper architecture design, neglecting to explore the feature correlations of intermediate layers, hence hindering the representational power of CNNs. To address this issue, in this paper, we propose a second-order attention network (SAN) for more powerful feature expression and feature correlation learning. Specifically, a novel train- able second-order channel attention (SOCA) module is developed to adaptively rescale the channel-wise features by using second-order feature statistics for more discriminative representations. Furthermore, we present a non-locally enhanced residual group (NLRG) structure, which not only incorporates non-local operations to capture long-distance spatial contextual information, but also contains repeated local-source residual attention groups (LSRAG) to learn increasingly abstract feature representations. Experimental results demonstrate the superiority of our SAN network over state-of-the-art SISR methods in terms of both quantitative metrics and visual quality.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

CNN-Generated Images Are Surprisingly Easy to Spot… for Now (2019, methodology): “Interestingly, there are two models, SAN and DeepFake, where directly training on ProGAN without augmentation performs strongly ( 93 .”
Pre-Trained Image Processing Transformer (2020, neutral): “To verify the effectiveness of the proposed method, Urban100( × 4): img 004 HR VDSR[41] EDSR[51] RDN[94] OISR[35] SAN[17] RNAN [93] IGNN BSD68: 163085 GT Noisy ( σ =50) CBM3D [16] TNRD [14] RDN [94] DnCNN [87] MemNet [65] IRCNN [88] FFDNet [89] IPT (ours) we compare our results with various…”
Plug-and-Play Image Restoration With Deep Denoiser Prior (2020, methodology): “We note that SAN [87] .”
Dual Aggregation Transformer for Image Super-Resolution (2023, methodology): “We compare our models: DAT-S, DAT-2, and DAT, with state-of-the-art methods: EDSR [14], SRMDNF [27], RDN [32], OISR [9], RCAN [30], NLRN [15], RNAN [31], SRFBN [12], SAN [5], RFANet [16], IGNN [34], HAN [23], CSNLN [22], NLSA [21], CRAN [33], ELAN [29], DFSA [17], SwinIR [13], and CAT-A [4].”
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search (2020, result): “23 SAN [19] 38.”
Learned Image Downscaling for Upscaling Using Content Adaptive Resampler (2019, background): “[28], i . e ., the SRCNN, many state-of-the-art neural SR models have been proposed [29]–[40].”
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials (2022, methodology): “ProGAN [33] LSUN 736.0k StyleGAN [34] LSUN 12.0k BigGAN [7] ImageNet 4.0k CycleGAN [88] Style/object transfer 2.6k GauGAN [58] COCO 10.0k StarGAN [13] CelebA 16.8k Non-GAN Model Glow [37] CelebA 16.8k CRN [11] GTA 12.8k IMLE [43] GTA 12.8k SAN [15] Standard SR benchmark 440 FaceForensics++ [64] YouT”
Efficient Long-Range Attention Network for Image Super-resolution (2022, methodology): “In addition, to enlarge the receptive field, the CNN-based SR models tend to employ very deep and complicated network topology [60, 10, 64, 41, 40] to recover more details, resulting in much computational resource consumption.”
Real-World Super-Resolution via Kernel Estimation and Noise Injection (2020, methodology): “[8] propose a second-order channel attention module to adaptively rescale the channel-wise features by using second-order feature statistics for more discriminative representations.”
Multi-Stage Progressive Image Restoration (2021, methodology): “Numerous network modules and functional units for image restoration have been developed including recursive residual learning [4, 95], dilated convolutions [4, 81], attention mechanisms [17, 86, 96], dense connections [73, 75, 97], encoder-decoders [7, 13, 43, 65], and generative models [44, 62, 90,”
Towards Real-World Blind Face Restoration with Generative Facial Prior (2021, background): “Image Restoration typically includes super-resolution [14, 50, 62, 51, 75, 10, 23, 52], denoising [72, 44, 27], deblurring [67, 40, 60] and compression removal [13, 22].”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, background): “lution branches, we also need a mechanism to share information within a feature tensor, both along the spatial and the channel dimensions. Motivated by the advances of recent low-level vision methods [125,5,23,126] based on the attention mechanisms [44,103], we propose the dual attention unit (DAU) ”
Unsupervised Degradation Representation Learning for Blind Super-Resolution (2021, background): “Recently, channel attention and second-order channel attention are further introduced by RCAN [44] and SAN [7] to exploit feature correlation for improved performance.”
Correction to: Single Image Super-Resolution via a Holistic Attention Network (2020, methodology): “Although the results obtained by using channel attention [40,2] retain some detailed information, these channel attention-based approaches struggle in”
Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution (2020, neutral): “917 SAN [8] 15.”
Transformer for Single Image Super-Resolution (2021, background): “SAN [9] presents a second-order attention network to enhance the feature expression and feature correlation learning.”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “Images A new dataset of CNN-generated images, namely ForenSynths, consisting of synthesized images from 11 models such as StyleGAN [51], super-resolution methods [161] and FaceForensics++ [105].”
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution (2020, background): “Many existing SR algorithms [4, 6, 7, 8, 12, 34, 42] directly learn a mapping from the lowresolution images to high-resolution images with a pixelwise constraint (e.”
Unfolding the Alternating Optimization for Blind Super Resolution (2020, methodology): “But most of these methods [39, 2, 40, 23, 8, 21] assume that the blur kernel is predefined as the kernel of bicubic interpolation.”
Residual Feature Aggregation Network for Image Super-Resolution (2020, methodology): “[3] proposed the second-order attention network (SAN) to explore more powerful feature expression by using second-order feature statistics.”