A Model-Driven Deep Neural Network for Single Image Rain Removal

By Hong Wang, Qi Xie, Qian Zhao, Deyu Meng. Computer Vision and Pattern Recognition, 2020

429 citations recorded by the source corpus.

Deep learning (DL) methods have achieved state-of-the-art performance in the task of single image rain removal. Most of current DL architectures, however, are still lack of sufficient interpretability and not fully integrated with physical structures inside general rain streaks. To this issue, in this paper, we propose a model-driven deep neural network for the task, with fully interpretable network structures. Specifically, based on the convolutional dictionary learning mechanism for representing rain, we propose a novel single image deraining model and utilize the proximal gradient descent technique to design an iterative algorithm only containing simple operators for solving the model. Such a simple implementation scheme facilitates us to unfold it into a new deep network architecture, called rain convolutional dictionary network (RCDNet), with almost every network module one-to-one corresponding to each operation involved in the algorithm. By end-to-end training the proposed RCDNet, all the rain kernels and proximal operators can be automatically extracted, faithfully characterizing the features of both rain and clean background layers, and thus naturally lead to its better deraining performance, especially in real scenarios. Comprehensive experiments substantiate the superiority of the proposed network, especially its well generality to diverse testing scenarios and good interpretability for all its modules, as compared with state-of-the-arts both visually and quantitatively.

Subfields: Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/e789c737d2c374a0c4c372ff5e218ed2e86ca0ee

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Risk, Reliability and Quality

Citations in the corpus

Learning A Sparse Transformer Network for Effective Image Deraining (2023, methodology): “We compare our DRSformer with two prior-based models (DSC [30] and GMM [24]), CNN-based methods (DDN [8], RESCAN [23], PReNet [36], MSPFN [19], RCDNet [43], MPRNet [59], DualGCN [9], and SPDNet [56]), and recent Transformer-based methods (Uformer [48], Restormer [58], and IDT [50]).”
Image De-raining via Continual Learning (2021, background): “In recent years, remarkable progress has been achieved on single image de-raining and other low-level vision tasks due to the rapid growth of deep learning [32, 9, 5, 35, 45, 18, 50, 43, 27, 8, 44, 41, 33, 23, 13, 37, 3, 6, 30].”
Uformer: A General U-Shaped Transformer for Image Restoration (2021, methodology): “We conduct the deraining experiments on SPAD [61] and compare with 6 deraining methods: GMM [34], RESCAN [33], SPANet [61], JORDER-E [71], RCDNet [60], and SPAIR [47].”
Learning to Optimize: A Primer and A Benchmark (2021, neutral): “Popular application examples include denoising [63; 172; 256; 51; 192; 126; 234; 134; 142], deblurring [63; 256; 146; 127; 58; 142; 162; 257; 121], super-resolution [175; 72; 172; 256; 51; 127; 257], inpainting [175; 172; 192; 3], and compressive sensing [175; 153; 43; 63; 171; 141; 255; 98; 142], J”
Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness (2021, background): “There are also other frameworks proposed for many other low-level tasks, such as image rain removal [42]–[44] and super-resolution [45]–[47].”
Image De-Raining Transformer (2022, methodology): “There is a tendency to integrate traditional methods with deep learning approaches for taking advantage of image priors and powerful feature mapping [74], [75].”
All-In-One Image Restoration for Unknown Corruption (2022, background): “Other image restoration tasks have also faced the similar challenge, such as debluring [2,12,29,32–34,36], deraining [10,17,24,42,46,49, 50,52], and dehazing [1,15,20,25,28,35,37,38].”
TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, background): “Adverse weather removal problems like deraining [16, 21,25,51,61,65,76], dehazing [1,2,10,22,42,69], desnowing [29,44,44,73] and rain drop removal [37,39,40,66] have been extensively explored in the literature.”