Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal

By Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, John William Paisley. IEEE Transactions on Image Processing, 2016

906 citations recorded by the source corpus.

We introduce a deep network architecture called DerainNet for removing rain streaks from an image. Based on the deep convolutional neural network (CNN), we directly learn the mapping relationship between rainy and clean image detail layers from data. Because we do not possess the ground truth corresponding to real-world rainy images, we synthesize images with rain for training. In contrast to other common strategies that increase depth or breadth of the network, we use image processing domain knowledge to modify the objective function and improve deraining with a modestly sized CNN. Specifically, we train our DerainNet on the detail (high-pass) layer rather than in the image domain. Though DerainNet is trained on synthetic data, we find that the learned network translates very effectively to real-world images for testing. Moreover, we augment the CNN framework with image enhancement to improve the visual results. Compared with the state-of-the-art single image de-raining methods, our method has improved rain removal and much faster computation time after network training.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/93c49234abcf6ab8ac82140ae3ce5af797e9132b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Depth-Attentional Features for Single-Image Rain Removal (2019, methodology): “In the literature on single-image rain removal, existing methods focused on removing the rain streaks by adopting various image priors [5, 19, 22, 23, 31, 32, 41, 52], or by exploiting a deep convolutional neural network (CNN) to learn a mapping between the training images with and without rain stre”
An Overview of Autonomous Vehicles Sensors and Their Vulnerability to Weather Conditions (2021, neutral): “Fortunately, some digital image processing techniques can mitigate the effect of precipitation and improve images quality under dynamic weather conditions [38].”
Progressive Image Deraining Networks: A Better and Simpler Baseline (2019, methodology): “In this paper, we aim to present a new baseline network for single image deraining to demonstrate that: (i) by combining only a few modules, a better and simpler baseline network can be constructed and achieve noteworthy performance gains over state-of-the-art deeper and complex deraining networks, ”
Lightweight Pyramid Networks for Image Deraining (2018, methodology): “[30], [31] use the guided filter [38] to”
A Model-Driven Deep Neural Network for Single Image Rain Removal (2020, methodology): “Driven by the significant success of deep learning (DL) in low level vision, recent years have also witnessed the rapid progress of deep convolutional neural networks (CNN) for single image rain removal [8, 52, 53, 40].”
Visual Quality Enhancement Of Images Under Adverse Weather Conditions (2018, background): “Another recent deep learning based rain removal is introduced by Fu et al. [18] where the removal of rain streaks is done on the high frequency detail layer by learning a bank of ﬁlters.”
Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study (2019, background): “In [15, 16], deep networks take the image detail”
SAR Image Despeckling Using a Convolutional Neural Network (2017, background): “In recent years, deep-learning-based methods have shown to produce state-of-the-art results on various image processing tasks such are image restoration [15], [16] and super resolution [17].”
Perceptual Adversarial Networks for Image-to-Image Transformation (2017, background): “Image de-raining (or de-snowing) methods attempt to remove the rain (or snow) strikes in the pictures brought by the uncontrollable weather conditions [6], [24], [28].”
Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining (2019, methodology): “We conducted experiments on the real-world images provided by [34, 7, 35].”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, background): “Reference Rainy DerainNet [21] SEMI [81] UMRL [87]”
Deep Generalized Unfolding Networks for Image Restoration (2022, methodology): “We compare our proposed DGUNet with eight competitive methods [19,31,38,54,68,72,79,81].”
Density-Aware Single Image De-raining Using a Multi-stream Dense Network (2018, methodology): “Various methods have been proposed in the literature to address this problem [19, 7, 39, 21, 3, 16, 11, 2, 41, 36, 6].”
Single Image Deraining: A Comprehensive Benchmark Analysis (2019, background): “Data-driven CNN models: Recently, CNNs have achieved dominant success for image restoration [28, 29] including single image deraining [30, 31].”
Rain Removal in Traffic Surveillance: Does it Matter? (2018, methodology): “4) Convolutional Neural Networks: Convolutional Neural Networks (CNNs) were introduced to rain streak removal almost simultaneously in [2], [3], [23], and [24].”
Pre-Trained Image Processing Transformer (2020, methodology): “Input DSC [28] GMM [49] JCAS [31] Clear [27] DDN [ demonstrates that the pre-trained model can capture more useful information and features from the large-scale dataset.”
The Impact of Adverse Weather Conditions on Autonomous Vehicles: How Rain, Snow, Fog, and Hail Affect the Performance of a Self-Driving Car (2019, methodology): “For posttime processing, the authors in [11] used the deep convolutional neural network to remove rain streaks from an image.”