TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions
By Jeya Maria Jose Valanarasu, Rajeev Yasarla, Vishal M. Patel. Computer Vision and Pattern Recognition, 2021
524 citations recorded by the source corpus.
Removing adverse weather conditions like rain, fog, and snow from images is an important problem in many applications. Most methods proposed in the literature have been designed to deal with just removing one type of degradation. Recently, a CNN-based method using neural architecture search (All-in-One) was proposed to remove all the weather conditions at once. However, it has a large number of parameters as it uses multiple encoders to cater to each weather removal task and still has scope for improvement in its performance. In this work, we focus on developing an efficient solution for the all adverse weather removal problem. To this end, we propose TransWeather, a transformer-based end-to-end model with just a single encoder and a decoder that can restore an image degraded by any weather condition. Specifically, we utilize a novel transformer encoder using intra-patch transformer blocks to enhance attention inside the patches to effectively remove smaller weather degradations. We also introduce a transformer decoder with learnable weather type embeddings to adjust to the weather degradation at hand. Trans Weather achieves significant improvements across multiple test datasets over both All-in-One network as well as methods fine-tuned for specific tasks. TransWeather is also validated on real world test images and found to be more effective than previous methods. Implementation code can be found in the supplementary document. Code is available at https//github.com/jeya-maria-jose/TransWeather.
Subfields: Computer Vision and Pattern Recognition, Media Technology
View Paper: https://www.semanticscholar.org/paper/b27d3be4264dcd06f990b44968f4382526f24f1e
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
In Media Technology
Citations in the corpus
- Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models (2022, methodology): “We compare WeatherDiffp with two state-of-the-art multi-weather image restoration methods: All-in-One [9], which utilizes a multi-encoder and decoder pipeline with a neural architecture search mechanism, and TransWeather [7], which exploits an end-to-end vision transformer.”
- Vision Transformers for Single Image Dehazing (2022, unknown): “However, HyLoG-ViT does not show convincing performance, while TransWeather aims to use a DETR-like framework [65] to handle multiple weather conditions simultaneously.”