Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining

By Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, Hongyuan Zha. 2018

892 citations recorded by the source corpus.

Rain streaks can severely degrade the visibility, which causes many current computer vision algorithms fail to work. So it is necessary to remove the rain from images. We propose a novel deep network architecture based on deep convolutional and recurrent neural networks for single image deraining. As contextual information is very important for rain removal, we first adopt the dilated convolutional neural network to acquire large receptive field. To better fit the rain removal task, we also modify the network. In heavy rain, rain streaks have various directions and shapes, which can be regarded as the accumulation of multiple rain streak layers. We assign different alpha-values to various rain streak layers according to the intensity and transparency by incorporating the squeeze-and-excitation block. Since rain streak layers overlap with each other, it is not easy to remove the rain in one stage. So we further decompose the rain removal into multiple stages. Recurrent neural network is incorporated to preserve the useful information in previous stages and benefit the rain removal in later stages. We conduct extensive experiments on both synthetic and real-world datasets. Our proposed method outperforms the state-of-the-art approaches under all evaluation metrics. Codes and supplementary material are available at our project webpage: https://xialipku.github.io/RESCAN.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising (2019, positive): “Following several recent image restoration networks [18, 31, 14], we do not reduce the spatial resolution of the input.”
Gated Context Aggregation Network for Image Dehazing and Deraining (2018, methodology): “Since dilated convolution is widely used to aggregate context information for its effectiveness without sacriﬁcing the spatial resolution [41, 25, 36, 15, 9], we also adopt it to help obtain more accurate restoration results by covering more neighbor pixels.”
Image De-raining via Continual Learning (2021, background): “Further, more complicated CNN-based architectures are designed to improve the performance on rain removal, including non-local operation based encoderdecoder network [22], multi-stages network [47], conditional generative adversarial network [48], deep convolutional and recurrent neural network that”
Deep Comprehensive Correlation Mining for Image Clustering (2019, background): “Recently, with the booming of deep learning [28, 21, 46, 30, 51], many researchers shift their attention to deep unsupervised feature learning and clustering [42, 23, 8], which can well solve the aforementioned limitations.”
A Model-Driven Deep Neural Network for Single Image Rain Removal (2020, methodology): “Typical deraining network structures include deep detail network (DDN) [9], recurrent squeeze-and-excitation context aggregation module (RESCAN) [27], progressive image deraining network (PReNet) [35], spatial attentive unit (SPANet) [41], and many others.”
All in One Bad Weather Removal Using Architectural Search (2020, methodology): “The baseline methods for deraining comprise of the state-of-the-art heavy rain removal HRGAN [25], RESCAN [29], and DetailsNet [8].”
Image De-Raining Transformer (2022, background): “Li et al. [70] designed a multi-stage recurrent network incorporating the squeeze-and-excitation block [71] to remove rain streaks.”
All-In-One Image Restoration for Unknown Corruption (2022, background): “Other image restoration tasks have also faced the similar challenge, such as debluring [2,12,29,32–34,36], deraining [10,17,24,42,46,49, 50,52], and dehazing [1,15,20,25,28,35,37,38].”
Depth-Attentional Features for Single-Image Rain Removal (2019, methodology): “Among them, DID-MDN [50] and RESCAN [30] use deep neural networks to recover the background by learning the mapping function between the rain and rain-free images from the training data, while others employ the hand-crafted priors to remove rain streaks.”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, methodology): “Rainy Image RESCAN [43] PreNet [64] MSPFN [32] MPRNet [93] Restormer”
Multi-Stage Progressive Image Restoration (2021, methodology): “These methods [24, 47, 53, 63, 70, 71, 88, 99] aim to recover clean image in a progressive manner by employing a light-weight subnetwork at each stage.”
CLEARER: Multi-Scale Neural Architecture Search for Image Restoration (2020, methodology): “From the left to the right are Input, DSC [26], LP [18], DetailsNet [6], JORDER [45], SCAN [17], RESCAN [17], CLEARER, and Ground truth.”
Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning (2019, methodology): “Principally, Figure 1: A comparison of our algorithm with combined state of the art dehazing/defogging [2] and deraining [21][40].”
LatticeNet: Towards Lightweight Image Super-Resolution with Lattice Block (2020, background): “The attention module is introduced for low-level image restoration in [36, 24].”
Cooperative Semantic Segmentation and Image Restoration in Adverse Environmental Conditions (2019, neutral): “Table 3 shows the comparison results with the state- of-the-art rain removal methods, i.e., DSC [26], GMMLP [22], JOB [54], RESCAN [20], DID-MDN [49], SPANet [41], PReNet [32] and DAF-Net [14].”
Expectation-Maximization Attention Networks for Semantic Segmentation (2019, methodology): “To capture long-range dependencies, several works employ the multi-scale context fusion [17], such as astrous convolution [4], spatial pyramid [37], large kernel convolution [25] and so on.”
Progressive Image Deraining Networks: A Better and Simpler Baseline (2019, methodology): “Besides, we have tried to train RESCAN for light rainy images, but the results are much inferior to the others.”
Multi-Scale Progressive Fusion Network for Single Image Deraining (2020, methodology): “…select a total of 850 samples from COCO [3] and BDD [38] datasets to create three new synthetic rain datasets COCO350 (for detection), BDD350 (for performing deraining methods [19, 26], the restoration procedures are directly implemented on these three datasets to produce the rain-free images.”