Attentive Generative Adversarial Network for Raindrop Removal from A Single Image

By Robby T. Tan, Wenhan Yang, Jiajun Su, Jiaying Liu. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

809 citations recorded by the source corpus.

Raindrops adhered to a glass window or camera lens can severely hamper the visibility of a background scene and degrade an image considerably. In this paper, we address the problem by visually removing raindrops, and thus transforming a raindrop degraded image into a clean one. The problem is intractable, since first the regions occluded by raindrops are not given. Second, the information about the background scene of the occluded regions is completely lost for most part. To resolve the problem, we apply an attentive generative network using adversarial training. Our main idea is to inject visual attention into both the generative and discriminative networks. During the training, our visual attention learns about raindrop regions and their surroundings. Hence, by injecting this information, the generative network will pay more attention to the raindrop regions and the surrounding structures, and the discriminative network will be able to assess the local consistency of the restored regions. This injection of visual attention to both generative and discriminative networks is the main contribution of this paper. Our experiments show the effectiveness of our approach, which outperforms the state of the art methods quantitatively and qualitatively.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/34e38559df559bfedfc3e1edc2b120ab2b5d444e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, methodology): “For rain drop removal, we compare the performance with state-ofthe-art methods like Attention GAN [37], Quan et al.”
Self-derived organ attention for unpaired CT-MRI deep domain adaptation based MRI segmentation (2020, methodology): “Our idea of using organ attention is somewhat similar to self-attention based I2I translations previously used for analyzing natural images for the purpose of image editing (Qian et al 2018), and in classification (Wang et al 2017).”
EnlightenGAN: Deep Light Enhancement Without Paired Supervision (2019, methodology): “When applying GANs to image restoration and enhancement, most existing works use paired training data as well, such as super resolution [27], single image deblurring [28], deraining [29] and dehazing [30].”
Depth-Attentional Features for Single-Image Rain Removal (2019, background): “, snow [36], raindrops [34, 48], and haze [3, 27, 29, 35, 46, 49].”
Progressive Image Deraining Networks: A Better and Simpler Baseline (2019, background): “And adversarial loss is also introduced to enhance the texture details of deraining result [25, 33].”
All in One Bad Weather Removal Using Architectural Search (2020, methodology): “Self-Attention Operation A few methods [41, 42] have demonstrated the advantages of using an attention map for removing adherent raindrops.”
Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks (2020, background): “There are also some recent works on bad weather restoration or image enhancement, such as dehazing [36], [37], raindrop and dirt removal [38], [39], light enhancement [40], [41] and moderate rain removal [1], [22], [42], [43].”
ADCrowdNet: An Attention-Injective Deformable Convolutional Network for Crowd Understanding (2018, background): “This principle of focusing on specific parts of the input has been successfully applied in various deep learning models for images classification [11], semantic segmentation [24], image deblurring [22], and visual pose estimation [6], which also suits our problem where the interest regions containin”
Single Image Deraining: A Comprehensive Benchmark Analysis (2019, methodology): “Degraded GMM [27] JORDER [32] DDN [6] CGAN [33] DID-MDN [8] DeRaindrop [7]”
Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models (2022, methodology): “RainDrop [12] is a dataset of images with raindrops introducing artifacts on the camera sensor and obstructing the view.”
Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning (2019, methodology): “For the training of the model-free refinement stage, we create another outdoor rain dataset on a set of outdoor clean images from [28], denoted as Outdoor-Rain.”
Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration (2019, methodology): “It is seen that the results of our method are visually comparable to the method of [30].”
Image De-Raining Transformer (2022, methodology): “For raindrop removal, we compare against Eigen’s model [104], Pix2Pix [105], AttentGAN [102] and Quan’s network[106].”
Deep Retinex Decomposition for Low-Light Enhancement (2018, methodology): “With the rapid development of deep neural network, CNN has been widely used in lowlevel image processing, including super-resolution [6, 24, 26, 27], rain removal [16, 21, 25] et al.”
MAXIM: Multi-Axis MLP for Image Processing (2022, methodology): “13, our model achieved the best performance: 31.87 dB and 25.74 dB PSNR on Raindrop testset A and B.”