Density-Aware Single Image De-raining Using a Multi-stream Dense Network

By He Zhang, Vishal M. Patel. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

1,052 citations recorded by the source corpus.

Single image rain streak removal is an extremely challenging problem due to the presence of non-uniform rain densities in images. We present a novel density-aware multi-stream densely connected convolutional neural network-based algorithm, called DID-MDN, for joint rain density estimation and de-raining. The proposed method enables the network itself to automatically determine the rain-density information and then efficiently remove the corresponding rain-streaks guided by the estimated rain-density label. To better characterize rain-streaks with different scales and shapes, a multi-stream densely connected de-raining network is proposed which efficiently leverages features from different scales. Furthermore, a new dataset containing images with rain-density labels is created and used to train the proposed density-aware network. Extensive experiments on synthetic and real datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods. In addition, an ablation study is performed to demonstrate the improvements obtained by different modules in the proposed method. The code can be downloaded at https://github.com/hezhangsprinter/DID-MDN

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/34aaa735409b666b8dc678c23acb600b1d87913b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset (2019, methodology): “(d) DDN [11] (e) DID-MDN [42] (f) RESCAN [25]”
Residual Dense Network for Image Super-Resolution (2018, methodology): “Recently, deep learning (DL)-based methods have achieved dramatic advantages against conventional methods in computer vision [36, 35, 34, 16].”
Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning (2019, methodology): “This applies to all the image-based [16, 24, 14, 22, 38, 7, 40, 21] and video-based works [41, 8, 1, 3, 23, 17, 6, 19, 31, 5, 32, 33, 39].”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, background): “From level-1 to level-4, the number of Transformer blocks are [4, 6, 6, 8], attention heads in MDTA are [1, 2, 4, 8], and number of channels are [48, 96, 192, 384].”
Tell Me Where to Look: Guided Attention Inference Network (2018, neutral): “Since deep neural networks have achieved great success in many areas [7, 34, 35, 37], various methods have been proposed to explain this black box [3, 26, 33, 38].”
Deep Generalized Unfolding Networks for Image Restoration (2022, methodology): “We compare our proposed DGUNet with eight competitive methods [19,31,38,54,68,72,79,81].”
All-In-One Image Restoration for Unknown Corruption (2022, background): “To be specific, the denoising baselines contain CBM3D [5], DnCNN [53], IRCNN [54], FFDNet [55] and BRDNet [40]. the deraining baselines are DIDMDN [52], UMRL [50], SIRR [46], MSPFN [17], and LPNet [11].”
Gated Context Aggregation Network for Image Dehazing and Deraining (2018, methodology): “Note that all the results are cited from [43].”
Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks (2020, methodology): “9 clearly shows that, DetailNet and JORDER are signiﬁcantly better than LP [21], DID-MDN [65], ID-CGAN [66], JCAS [64], and UGSM [63].”
Densely Connected Pyramid Dehazing Network (2018, methodology): “Based on these observations and inspired by the gradient loss used in depth estimation [43, 26] as well as the use of perceptual loss in low-level vision tasks [20, 49], we propose a new edge-preserving loss function that is composed of three different parts: L2 loss, two-directional gradient loss, ”
Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration (2019, background): “[52] proposed to jointly estimate rain density and de-raining result to alleviate the non-uniform rain density problem.”
All in One Bad Weather Removal Using Architectural Search (2020, background): “This is also true for bad weather restoration or image enhancement, such as dehazing [47, 3, 1, 57, 7, 13, 26, 43], removal of raindrop and dirt [20, 18, 59, 8, 53, 30, 50, 62, 36, 4], of moderate rain [44, 29], and of heavy rain [25, 54].”
Image De-raining via Continual Learning (2021, background): “Further, more complicated CNN-based architectures are designed to improve the performance on rain removal, including non-local operation based encoderdecoder network [22], multi-stages network [47], conditional generative adversarial network [48], deep convolutional and recurrent neural network that”