Plug-and-Play Image Restoration With Deep Denoiser Prior

By Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, Radu Timofte. 2020

1,208 citations recorded by the source corpus.

Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems. Such a property induces considerable advantages for plug-and-play image restoration (e.g., integrating the flexibility of model-based method and effectiveness of learning-based methods) when the denoiser is discriminatively learned via deep convolutional neural network (CNN) with large modeling capacity. However, while deeper and larger CNN models are rapidly gaining popularity, existing plug-and-play image restoration hinders its performance due to the lack of suitable denoiser prior. In order to push the limits of plug-and-play image restoration, we set up a benchmark deep denoiser prior by training a highly flexible and effective CNN denoiser. We then plug the deep denoiser prior as a modular part into a half quadratic splitting based iterative algorithm to solve various image restoration problems. We, meanwhile, provide a thorough analysis of parameter setting, intermediate results and empirical convergence to better understand the working mechanism. Experimental results on three representative image restoration tasks, including deblurring, super-resolution and demosaicing, demonstrate that the proposed plug-and-play image restoration with deep denoiser prior not only significantly outperforms other state-of-the-art model-based methods but also achieves competitive or even superior performance against state-of-the-art learning-based methods. The source code is available at https://github.com/cszn/DPIR.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, methodology): “As mentioned in the introduction, existing DNNs-based SISR methods are generally based on bicubic downsampling [23, 44] and traditional degradations [26, 37, 54, 59, 60], or some simple variants [11, 41, 53, 56, 58].”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, methodology): “Among convolutional designs, encoder-decoder based UNet architectures [3, 14, 39, 80, 90, 93, 99] have been predominantly studied for restoration due to their hierarchical multi-scale representation while remaining computationally efficient.”
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration (2023, methodology): “7% of DRUNet [80]) while not sacrificing network accuracy.”
Efficient Long-Range Attention Network for Image Super-resolution (2022, neutral): “Dataset RPCNN [60] BRDNet [52] DRUNet [63] SwinIR [29] ELAN”
MambaIR: A Simple Baseline for Image Restoration with State-Space Model (2024, neutral): “Following [79,81], the compared noise levels include 15, 25 and 50.”
SwinIR: Image Restoration Using Swin Transformer (2021, methodology): “Following [98, 88], we test different methods on two benchmark datasets (Classic5 [22] and LIVE1 [67]) for JPEG quality factors 10, 20, 30 and 40.”
Denoising Diffusion Models for Plug-and-Play Image Restoration (2023, methodology): “…have demonstrated that plug-and-play Image Restoration (IR) methods can effectively handle a variety of low-level vision tasks, such as image denoising [5,58], image Super-Resolution (SR) [16,17,37], image deblurring [12] and image inpainting [27], with excellent results [6, 11, 54, 57, 59].”
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (2021, background): “Since the pioneer work SRCNN [9], many CNNbased models have been proposed in recent years [9, 19, 24, 43, 41, 40, 42, 15, 6, 6, 54, 55, 30, 49, 11, 5, 44, 26, 51, 27, 50].”
DiffIR: Efficient Diffusion Model for Image Restoration (2023, neutral): “0001 for DiffIR and discriminator networks, respectively.”
Image De-Raining Transformer (2022, background): “In recent years, deep learning are widely used in various image processing tasks [17], [18], [31], [32], [33], [34], [35], such as de-noising [36], [37], [38], super-resolution [39], [40], [41], de-blurring [42], [43], [44], [45], de-hazing [19], [19], [46], [47], [48], [49], [50], low-light image…”