Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset

By Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, Rynson W. H. Lau. Computer Vision and Pattern Recognition, 2019

677 citations recorded by the source corpus.

Removing rain streaks from a single image has been drawing considerable attention as rain streaks can severely degrade the image quality and affect the performance of existing outdoor vision tasks. While recent CNN-based derainers have reported promising performances, deraining remains an open problem for two reasons. First, existing synthesized rain datasets have only limited realism, in terms of modeling real rain characteristics such as rain shape, direction and intensity. Second, there are no public benchmarks for quantitative comparisons on real rain images, which makes the current evaluation less objective. The core challenge is that real world rain/clean image pairs cannot be captured at the same time. In this paper, we address the single image rain removal problem in two ways. First, we propose a semi-automatic method that incorporates temporal priors and human supervision to generate a high-quality clean image from each input sequence of real rain images. Using this method, we construct a large-scale dataset of ∼29.5K rain/rain-free image pairs that covers a wide range of natural rain scenes. Second, to better cover the stochastic distribution of real rain streaks, we propose a novel SPatial Attentive Network (SPANet) to remove rain streaks in a local-to-global manner. Extensive experiments demonstrate that our network performs favorably against the state-of-the-art deraining methods.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/d3fa312b1f12ee60391705d3cac34cbcad42db14

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jiayi Ma
2. Vishal M. Patel
3. Luc Van Gool
4. Linfeng Tang
5. Yunhe Wang
6. Yong Ma
7. Rajeev Yasarla
8. Jin-shan Pan

In Media Technology

1. Jiayi Ma
2. Linfeng Tang
3. Yong Ma
4. Jun Huang
5. Junjun Jiang
6. Yuxin Deng
7. Robert A. Legenstein
8. Ozan Özdenizci

Citations in the corpus

• Multi-Scale Progressive Fusion Network for Single Image Deraining (2020, methodology): “Very recently, deep-learning based approaches [24, 29, 39] have emerged for rain streak removal and demonstrated impressive restoration performance.”
• Cooperative Semantic Segmentation and Image Restoration in Adverse Environmental Conditions (2019, neutral): “Table 3 shows the comparison results with the state- of-the-art rain removal methods, i.e., DSC [26], GMMLP [22], JOB [54], RESCAN [20], DID-MDN [49], SPANet [41], PReNet [32] and DAF-Net [14].”
• Learning A Sparse Transformer Network for Effective Image Deraining (2023, methodology): “In addition, we also evaluate our method using a large-scale real-world dataset, i.e , SPA-Data [45], containing 638,492 image pairs for training and 1,000 testing ones.”
• Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models (2022, methodology): “We perform comparisons of our weather-specific models with several state-of-the-art methods discussed in Section II-B for image desnowing [3], [43], [44], [58], [59], combined image deraining and dehazing [14], [45], [46], [53], [54], and removing raindrops [6], [12], [45], [55], [56].”
• SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness (2022, methodology): “[71] introduced two-round four-directional RNN to the shadow removal and single-image deraining tasks, respectively.”
• Image De-raining via Continual Learning (2021, methodology): “In detail, PreNet, PRN, NLEDN, REHEN and SASI are trained on Rain100H (Task 0) and Rain100L (Task 1) sequentially.”
• TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, background): “Recently, Convolutional Neural Networks (CNNs) based solutions have been explored extensively for deraining [11, 37, 53, 56, 61, 63, 69, 70, 76], dehazing [8, 19, 41, 57, 69, 71, 72], desnowing [29, 44, 73] and raindrop removal [37, 40, 66].”
• Pre-Trained Image Processing Transformer (2020, background): “A various excellent works are also proposed for the other tasks, such as denoising [68, 32, 37, 45, 24], dehazing [6, 46, 85, 80], deraining [36, 78, 62, 29, 74, 47], and debluring [67, 53, 23, 10].”
• Image De-Raining Transformer (2022, methodology): “In order to improve realism of rain datasets, Wang et al. [73] constructed a large-scale real-world dataset and proposed a spatial attentive network to remove rain streaks in a local-to-global manner.”
• Uformer: A General U-Shaped Transformer for Image Restoration (2021, methodology): “More results on GoPro [58] for image motion deblurring.”