Learning Enriched Features for Real Image Restoration and Enhancement

By Syed Waqas Zamir, Aditya Arora, Salman Hameed Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao. European Conference on Computer Vision, 2020

989 citations recorded by the source corpus.

With the goal of recovering high-quality image content from its degraded version, image restoration enjoys numerous applications, such as in surveillance, computational photography, medical imaging, and remote sensing. Recently, convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. Existing CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatially precise but contextually less robust results are achieved, while in the latter case, semantically reliable but spatially less accurate outputs are generated. In this paper, we present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network and receiving strong contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing several key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) spatial and channel attention mechanisms for capturing contextual information, and (d) attention based multi-scale feature aggregation. In a nutshell, our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on five real image benchmark datasets demonstrate that our method, named as MIRNet, achieves state-of-the-art results for a variety of image processing tasks, including image denoising, super-resolution, and image enhancement. The source code and pre-trained models are available at this https URL.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/26f27bd5892e869b54c69d140b20c33638f26c57

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Uformer: A General U-Shaped Transformer for Image Restoration (2021, methodology): “For example, squeeze-and-excitation networks [20] and non-local neural networks [63] inspire a branch of methods for different image restoration tasks, such as super-resolution [36, 82], deraining [33, 61], denoising [76, 77], deblurring [47, 77], deshadow [11, 12], and so on.”
A Deep Journey into Super-resolution (2019, background): “Another recent effort proposes to enrich features by preserving the original spatial resolution and exchanging multi-scale information along the feature processing path for real image SR [109].”
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement (2023, methodology): “With the rapid progress of deep learning, CNN [15, 16, 21, 22, 25, 34, 36, 38, 45, 48, 60, 65, 67] has been widely used in low-light image enhancement.”
Deep Generalized Unfolding Networks for Image Restoration (2022, background): “Real image denoising performance (y-axis) of our DGUNet and some recent denoisers (VDNet [75], GDANet [76], AINDNet [33], MIRNet [78], DeamNet [53], MPRNet [79]) under different parameter capacities (x-axis) on DND [52] dataset.”
Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method (2022, methodology): “Same as (Zamir et al. 2020), we consider images from expert C for the MIT-Adobe FiveK dataset.”
MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (2022, methodology): “We also train MIRNet [84], MPRNet [85], Restormer [83], HINet [21], and MST [13] families.”
SNR-Aware Low-light Image Enhancement (2022, methodology): “…SOTA methods for low-light image enhancement, including Dong [9], LIME [15], MF [11], SRIE [12], BIMEF [55], DRD [45], RRM [28], SID [5], DeepUPE [40], KIND [61], DeepLPF [33], FIDE [48], LPNet [26], MIR-Net [59], RF [24], 3DLUT [60], A3DLUT [42], Band [52], EG [20], Retinex [29], and Sparse [53].”
Low-Light Image Enhancement with Normalizing Flow (2021, background): “Deep learning-based methods (Zhang et al. 2019; Zamir et al. 2020; Chen et al. 2018) achieve promising performance by utilizing the power of large collections of data.”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, methodology): “Method DnCNN BM3D CBDNet* RIDNet* AINDNet* VDN SADNet* DANet+* CycleISP* MIRNet DeamNet* MPRNet DAGL Uformer Restormer Dataset [101] [15] [25] [6] [35] [89] [12] [90] [91] [92] [63] [93] [55] [80] (Ours)”
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results (2020, methodology): “7: Framework of the network MIRNet (recently introduced in [39]) for the TeamInception team.”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, methodology): “In this work, we make several key modifications to MIRNet [9] that allow us to significantly reduce the computational cost while enhancing model performance (see Table 1).”
Low-Light Image Enhancement with Wavelet-Based Diffusion Models (2023, neutral): “…2) learning-based LLIE methods RetinexNet [Wei et al. 2018], DSLR [Lim and Kim 2020], DRBN [Yang et al. 2020a], Zero-DCE [Guo et al. 2020], MIRNet [Zamir et al. 2020], EnlightenGAN [Jiang et al. 2021], ReLLIE [Zhang et al. 2021], RUAS [Liu et al. 2021], SCI [Ma et al. 2022], URetinex-Net [Wu et…”
Image De-Raining Transformer (2022, methodology): “On the other hand, our work primarily focuses on establishing a transformer-based method for single image de-TABLE [114] Model BM3D[115] DnCNN[38] VDN[116] DANet[117] CycleISP[118] MIRNet[119] DeamNet[120] MPRNet[79] DAGL[121] Uformer-S raining.”