Semi-Supervised Transfer Learning for Image Rain Removal

By Wei Wei, Deyu Meng, Qian Zhao, Zongben Xu, Ying Wu. Computer Vision and Pattern Recognition, 2018

463 citations recorded by the source corpus.

Single image rain removal is a typical inverse problem in computer vision. The deep learning technique has been verified to be effective for this task and achieved state-of-the-art performance. However, previous deep learning methods need to pre-collect a large set of image pairs with/without synthesized rain for training, which tends to make the neural network be biased toward learning the specific patterns of the synthesized rain, while be less able to generalize to real test samples whose rain types differ from those in the training data. To this issue, this paper firstly proposes a semi-supervised learning paradigm toward this task. Different from traditional deep learning methods which only use supervised image pairs with/without synthesized rains, we further put real rainy images, without need of their clean ones, into the network training process. This is realized by elaborately formulating the residual between an input rainy image and its expected network output (clear image without rain) as a concise mixture of Gaussians distribution. The network is therefore trained to transfer to adapting the real rain pattern domain instead of only the synthesis rain domain, and thus both the short-of-training-sample and bias-to-supervised-sample issues can be evidently alleviated. Experiments on synthetic and real data verify the superiority of our model compared to the state-of-the-arts.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/a30b2a1b9080354bb507954d1ed87ad325a46c86

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

All-In-One Image Restoration for Unknown Corruption (2022, background): “To be specific, the denoising baselines contain CBM3D [5], DnCNN [53], IRCNN [54], FFDNet [55] and BRDNet [40]. the deraining baselines are DIDMDN [52], UMRL [50], SIRR [46], MSPFN [17], and LPNet [11].”
A Model-Driven Deep Neural Network for Single Image Rain Removal (2020, methodology): “[44] provided a semi-supervised rain removal method (SIRR) that described rain layer prior as a general GMM and jointly trained the backbone–DDN.”
Learning A Sparse Transformer Network for Effective Image Deraining (2023, unknown): “To better represent the rain distribution, several studies take rain characteristics such as rain direction [27], density [61], veiling effect [15] into account, and optimize the network structure via recursive computation [19, 23, 36] or transfer mechanism [16, 49, 54, 55].”
Deep Generalized Unfolding Networks for Image Restoration (2022, methodology): “We compare our proposed DGUNet with eight competitive methods [19,31,38,54,68,72,79,81].”
Image De-Raining Transformer (2022, background): “In addition, due to the lack of real-world fully-labeled images, semi-supervised learning-based meth-ods [76], [77], [78] are developed to improve the model generalization.”
Multi-Scale Progressive Fusion Network for Single Image Deraining (2020, methodology): “These representative methods include DerainNet [6], RESCAN [19], DIDMDN [40], UMRL [37], SEMI [31] and PreNet [26].”
PSD: Principled Synthetic-to-Real Dehazing Guided by Physical Priors (2021, background): “Specifically, in dehazing and deraining tasks, there have been a few studies to solve the domain shift problem between synthetic and real domains [39,41].”
TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, background): “Recently, Convolutional Neural Networks (CNNs) based solutions have been explored extensively for deraining [11, 37, 53, 56, 61, 63, 69, 70, 76], dehazing [8, 19, 41, 57, 69, 71, 72], desnowing [29, 44, 73] and raindrop removal [37, 40, 66].”
Semi-Supervised DAS VSP Data Denoising Using Signal and Noise Distribution Difference (2024, unknown): “Specifically, [36] introduces transfer learning to transfer from the synthesized rain domain to the real rain domain, and its semi-supervised learning idea reduces the domain gap between synthetic and real data.”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, background): “Reference Rainy DerainNet [21] SEMI [81] UMRL [87]”