All in One Bad Weather Removal Using Architectural Search

By Ruoteng Li, Robby T. Tan, Loong Fah Cheong. Computer Vision and Pattern Recognition, 2020

397 citations recorded by the source corpus.

Many methods have set state-of-the-art performance on restoring images degraded by bad weather such as rain, haze, fog, and snow, however they are designed specifically to handle one type of degradation. In this paper, we propose a method that can handle multiple bad weather degradations: rain, fog, snow and adherent raindrops using a single network. To achieve this, we first design a generator with multiple task-specific encoders, each of which is associated with a particular bad weather degradation type. We utilize a neural architecture search to optimally process the image features extracted from all encoders. Subsequently, to convert degraded image features to clean background features, we introduce a series of tensor-based operations encapsulating the underlying physics principles behind the formation of rain, fog, snow and adherent raindrops. These operations serve as the basic building blocks for our architectural search. Finally, our discriminator simultaneously assesses the correctness and classifies the degradation type of the restored image. We design a novel adversarial learning scheme that only backpropagates the loss of a degradation type to the respective task-specific encoder. Despite being designed to handle different types of bad weather, extensive experiments demonstrate that our method performs competitively to the individual and dedicated state-of-the-art image restoration methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2e89dd08b5ea206128724419e259c8750e8d1381

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, methodology): “Multi Task All-in-One [23] CVPR 2020 24.”
All-In-One Image Restoration for Unknown Corruption (2022, methodology): “On the one hand, the methods [3, 8, 23] have to specify the corruption type and ratio, whereas our method does not.”
Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models (2022, methodology): “We compare WeatherDiffp with two state-of-the-art multi-weather image restoration methods: All-in-One [9], which utilizes a multi-encoder and decoder pipeline with a neural architecture search mechanism, and TransWeather [7], which exploits an end-to-end vision transformer.”
ALL Snow Removed: Single Image Desnowing Algorithm Using Hierarchical Dual-tree Complex Wavelet Representation and Contradict Channel Loss (2021, background): “Dataset (size) Methods (PSNR/SSIM/CIEDE 2000) Zheng [5] Eigen [35] DesnowNet [1] CycleGAN [36] DAD [37] All in One [38] JSTASR [2] Ours Snow100K (2000) 23.86/0.82/3.90 20.12/0.72/6.40 30.50/0.94/2.31 26.81/0.89/5.73 24.93/0.84/7.61 26.07/0.88/5.53 23.12/0.86/6.31 31.54/0.95/2.18 SRRS (2000) wDH 19.1”
Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement (2020, methodology): “Very recently, NAS algorithms [24, 44, 14] have also been applied to lowlevel vision problems, such as image denoising, restoration, and deraining, etc.”