Residual Non-local Attention Networks for Image Restoration

By Yulun Zhang, Kunpeng Li, Kai Li, Bineng Zhong, Yun Raymond Fu. International Conference on Learning Representations, 2019

777 citations recorded by the source corpus.

In this paper, we propose a residual non-local attention network for high-quality image restoration. Without considering the uneven distribution of information in the corrupted images, previous methods are restricted by local convolutional operation and equal treatment of spatial- and channel-wise features. To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts. Specifically, we design trunk branch and (non-)local mask branch in each (non-)local attention block. The trunk branch is used to extract hierarchical features. Local and non-local mask branches aim to adaptively rescale these hierarchical features with mixed attentions. The local mask branch concentrates on more local structures with convolutional operations, while non-local attention considers more about long-range dependencies in the whole feature map. Furthermore, we propose residual local and non-local attention learning to train the very deep network, which further enhance the representation ability of the network. Our proposed method can be generalized for various image restoration applications, such as image denoising, demosaicing, compression artifacts reduction, and super-resolution. Experiments demonstrate that our method obtains comparable or better results compared with recently leading methods quantitatively and visually.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/34439db81b482cd562e1cdba974c70a2b89cd6d4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Residual Feature Aggregation Network for Image Super-Resolution (2020, background): “Following these pioneering works, many CNN-based methods have been proposed and achieved state-of-the-art results in SISR [15, 18, 40, 38, 3, 17, 7, 9, 39].”
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text (2021, background): “Early work studied hybrid models consisting of both convolutions and attention modules [94, 100, 40, 111].”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, methodology): “Motivated by the advances of recent low-level vision methods [32], [69], [70], [71] which incorporate attention mechanisms [110], [111], [112], we propose the residual contextual block (RCB) to extract features in the convolutional streams.”
Image De-Raining Transformer (2022, neutral): “In recent years, deep learning are widely used in various image processing tasks [17], [18], [31], [32], [33], [34], [35], such as de-noising [36], [37], [38], super-resolution [39], [40], [41], de-blurring [42], [43], [44], [45], de-hazing [19], [19], [46], [47], [48], [49], [50], low-light image…”
Spatial-Adaptive Network for Single Image Denoising (2020, positive): “RNAN [36] designed non-local attention blocks to capture global information and pay more attention to the challenging parts.”
Activating More Pixels in Image Super-Resolution Transformer (2022, methodology): “By using attention mechanism, [65,7,66,33,41,40] achieve further improvement in terms of reconstruction fidelity.”
Multi-Stage Progressive Image Restoration (2021, background): “Numerous network modules and functional units for image restoration have been developed including recursive residual learning [4, 95], dilated convolutions [4, 81], attention mechanisms [17, 86, 96], dense connections [73, 75, 97], encoder-decoders [7, 13, 43, 65], and generative models [44, 62, 90,”
Deep Learning for Image Super-Resolution: A Survey (2019, background): “[106] propose local and nonlocal attention blocks to extract features that capture the long-range dependencies between pixels.”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, background): “Similarly, skip connection based approaches have been shown to be effective for restoration due to specific focus on learning residual signals [24, 48, 92, 106].”
NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results (2020, methodology): “a RNAN [78] based SR network is trained.”
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, background): “The goal of IR is to convert degraded images into high-quality degradation-free images [22, 26, 86, 88, 95, 96].”
Plug-and-Play Image Restoration With Deep Denoiser Prior (2020, methodology): “, DnCNN [44], N(3)Net [60], NLRN [59], RNAN [69], FOCNet [70]) and two CNN-based methods which were trained to handle a wide range of noise levels (i.”
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules (2020, positive): “Furthermore, recent works use attention module to improve the performance for image restoration [29] and compression [30].”
Interpreting Super-Resolution Networks with Local Attribution Maps (2020, methodology): “Second, non-local and channelwise attention modules are introduced to SR networks in recent years, expecting to utilize long-term and global information to assist the SR. Representative network designs include RNAN [60], RCAN and SAN [13].”
All in One Bad Weather Removal Using Architectural Search (2020, methodology): “To overcome this problem, a selfattention mechanism [56, 48] has been applied to many image reconstruction methods [60, 33].”
SwinIR: Image Restoration Using Swin Transformer (2021, methodology): “Since pioneering work SRCNN [18] (for image SR), DnCNN [90] (for image denoising) and ARCNN [17] (for JPEG compression artifact reduction), a flurry of CNN-based models have been proposed to improve model representation ability by using larger and deeper neural network architecture designs, such as ”