Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning

By Ruoteng Li, Loong Fah Cheong, Robby T. Tan. Computer Vision and Pattern Recognition, 2019

459 citations recorded by the source corpus.

Most deraining works focus on rain streaks removal but they cannot deal adequately with heavy rain images. In heavy rain, streaks are strongly visible, dense rain accumulation or rain veiling effect significantly washes out the image, further scenes are relatively more blurry, etc. In this paper, we propose a novel method to address these problems. We put forth a 2-stage network: a physics-based backbone followed by a depth-guided GAN refinement. The first stage estimates the rain streaks, the transmission, and the atmospheric light governed by the underlying physics. To tease out these components more reliably, a guided filtering framework is used to decompose the image into its low- and high-frequency components. This filtering is guided by a rain-free residue image --- its content is used to set the passbands for the two channels in a spatially-variant manner so that the background details do not get mixed up with the rain-streaks. For the second stage, the refinement stage, we put forth a depth-guided GAN to recover the background details failed to be retrieved by the first stage, as well as correcting artefacts introduced by that stage. We have evaluated our method against state of the art methods. Extensive experiments show that our method outperforms them on real rain image data, recovering visually clean images with good details.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b1b7eeac57e1e78e02d9d97eb809d8459bd06050

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models (2022, methodology): “We perform comparisons of our weather-specific models with several state-of-the-art methods discussed in Section II-B for image desnowing [3], [43], [44], [58], [59], combined image deraining and dehazing [14], [45], [46], [53], [54], and removing raindrops [6], [12], [45], [55], [56].”
All in One Bad Weather Removal Using Architectural Search (2020, methodology): “We train our network on different bad weather datasets, including “Outdoor-Rain” [25], “Snow100K” [38], and “Raindrop” [41].”
TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, methodology): “We train Swin-IR and MPRNet directly on “Outdoor-Rain” (training split of Test1) and test it on Test1 for fair comparison.”
Image-to-Image Translation: Methods and Applications (2021, background): “Image deraining [195], [196], [197], image dehazing [199], [200], [201], [198], [202] and image deblurring [206], [203], [204], [205] aim to remove optical distortions from photos that were taken out of focus or while the camera was moving, or from photos of faraway geographical or astronomical feat”
Multi-Scale Progressive Fusion Network for Single Image Deraining (2020, background): “In the last few years, substantial improvements [24, 18, 4, 17] have been observed on rain image restoration.”
Image De-raining via Continual Learning (2021, background): “In recent years, remarkable progress has been achieved on single image de-raining and other low-level vision tasks due to the rapid growth of deep learning [32, 9, 5, 35, 45, 18, 50, 43, 27, 8, 44, 41, 33, 23, 13, 37, 3, 6, 30].”
Image De-Raining Transformer (2022, background): “There is a tendency to integrate traditional methods with deep learning approaches for taking advantage of image priors and powerful feature mapping [74], [75].”