Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks

By Wenhan Yang, Robby T. Tan, Jiashi Feng, Zongming Guo, Shuicheng Yan, Jiaying Liu. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

454 citations recorded by the source corpus.

Rain streaks, particularly in heavy rain, not only degrade visibility but also make many computer vision algorithms fail to function properly. In this paper, we address this visibility problem by focusing on single-image rain removal, even in the presence of dense rain streaks and rain-streak accumulation, which is visually similar to mist or fog. To achieve this, we introduce a new rain model and a deep learning architecture. Our rain model incorporates a binary rain map indicating rain-streak regions, and accommodates various shapes, directions, and sizes of overlapping rain streaks, as well as rain accumulation, to model heavy rain. Based on this model, we construct a multi-task deep network, which jointly learns three targets: the binary rain-streak map, rain streak layers, and clean background, which is our ultimate output. To generate features that can be invariant to rain steaks, we introduce a contextual dilated network, which is able to exploit regional contextual information. To handle various shapes and directions of overlapping rain streaks, our strategy is to utilize a recurrent process that progressively removes rain streaks. Our binary map provides a constraint and thus additional information to train our network. Extensive evaluation on real images, particularly in heavy rain, shows the effectiveness of our model and architecture.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/79890a7d61b082947e1300f60231336a53cc285c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Pre-Trained Image Processing Transformer (2020, methodology): “For the rain-streak generation, we follow the method described in [79].”
Lightweight Pyramid Networks for Image Deraining (2018, background): “Based on the introduction of an effective strategy for training very deep networks [20], two deeper networks are proposed based on image residuals [31] and multi-scale information [32], [33].”
TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, methodology): “Adverse weather removal problems like deraining [16, 21,25,51,61,65,76], dehazing [1,2,10,22,42,69], desnowing [29,44,44,73] and rain drop removal [37,39,40,66] have been extensively explored in the literature.”
All in One Bad Weather Removal Using Architectural Search (2020, background): “This is also true for bad weather restoration or image enhancement, such as dehazing [47, 3, 1, 57, 7, 13, 26, 43], removal of raindrop and dirt [20, 18, 59, 8, 53, 30, 50, 62, 36, 4], of moderate rain [44, 29], and of heavy rain [25, 54].”
A Model-Driven Deep Neural Network for Single Image Rain Removal (2020, methodology): “[49] developed a multi-stage joint rain detection and estimation network for single image (JORDER E).”
All-In-One Image Restoration for Unknown Corruption (2022, background): “For image deraining, we conduct experiments on Rain100L which consists of 200 rainy-clean training pairs and 100 testing image pairs.”
Image Restoration with Mean-Reverting Stochastic Differential Equations (2023, methodology): “Moreover, the CNN-baseline model only outperforms JORDER.”
A Survey of Deep Learning-Based Object Detection (2019, background): “[219] present a novel rain model accompany with a deep learning architecture to address rain detection in”
Image De-Raining Transformer (2022, background): “Yang et al. [67], [68] built a recurrent dilated network with multi-task learning for joint rain streaks detection and removal.”