WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images

By Jie Li, Katherine A. Skinner, Ryan M. Eustice, Matthew Johnson-Roberson. IEEE Robotics and Automation Letters, 2017

886 citations recorded by the source corpus.

This letter reports on WaterGAN, a generative adversarial network (GAN) for generating realistic underwater images from in-air image and depth pairings in an unsupervised pipeline used for color correction of monocular underwater images. Cameras onboard autonomous and remotely operated vehicles can capture high-resolution images to map the seafloor; however, underwater image formation is subject to the complex process of light propagation through the water column. The raw images retrieved are characteristically different than images taken in air due to effects, such as absorption and scattering, which cause attenuation of light at different rates for different wavelengths. While this physical process is well described theoretically, the model depends on many parameters intrinsic to the water column as well as the structure of the scene. These factors make recovery of these parameters difficult without simplifying assumptions or field calibration; hence, restoration of underwater images is a nontrivial problem. Deep learning has demonstrated great success in modeling complex nonlinear systems but requires a large amount of training data, which is difficult to compile in deep sea environments. Using WaterGAN, we generate a large training dataset of corresponding depth, in-air color images, and realistic underwater images. These data serve as input to a two-stage network for color correction of monocular underwater images. Our proposed pipeline is validated with testing on real data collected from both a pure water test tank and from underwater surveys collected in the field. Source code, sample datasets, and pretrained models are made publicly available.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6b8c9adb08da2d9b4357f9b3a4d9a3c7649ad06d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

1. Dacheng Tao
2. Lajos Hanzo
3. Junaed Sattar
4. Chunle Guo
5. Chongyi Li
6. Jichang Guo
7. M. Islam
8. Cameron Fabbri

In Computer Vision and Pattern Recognition

1. Chongyi Li
2. Chunle Guo
3. Chunli Zhu
4. Liheng Bian
5. Lintao Peng
6. Junaed Sattar
7. Peixian Zhuang
8. Runmin Cong

Citations in the corpus

• Underwater Image Enhancement via Medium Transmission-Guided Multi-Color Space Embedding (2021, methodology): “[11] employed a Generative Adversarial Network (GAN) and an image formation model to synthesize degraded/clean image pairs for supervised learning.”
• Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, neutral): “We have where The Downsampling module works as a feature map generator, who produces expanded features with lower dimensions.”
• Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception (2020, background): “Several GAN-based underwater image enhancement models have reported impressive results from both paired [20, 45] and unpaired training [35].”
• Underwater Image Enhancement via Weighted Wavelet Visual Perception Fusion (2024, methodology): “Inspired by GAN [53], [54], [55], Skinner et al. [56] constructed synthesized underwater images by the underwater physical model for GAN training, but the synthesized images are unable accurately represent the natural underwater scenes.”
• Fast Underwater Image Enhancement for Improved Visual Perception (2019, result): “based on deep adversarial [6], [18], [33] and residual learning [19] have reported inspiring results of late.”
• Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer (2017, background): “Recently, a semisupervised learning model for underwater image color correction, named WaterGAN, has been proposed [14].”
• A Generative Approach Towards Improved Robotic Detection of Marine Litter (2019, methodology): “Although there are generative model-based studies for the underwater domain, they mainly focus on image enhancements, such as WaterGAN [10], UGAN [18], and the synthesis of unpaired Underwater images using a Multistyle GAN (UMGAN) [11].”
• Underwater Image Enhancement With Hyper-Laplacian Reflectance Priors (2022, neutral): “WaterGAN [22] was proposed to correct the color Fig.”
• Internet of Underwater Things and Big Marine Data Analytics—A Comprehensive Survey (2020, methodology): “ically (for example by various data augmentation techniques or by employing generative neural networks, such as the Generative Adversarial Network (GAN) [313], [314]) or man-”
• UIEC^2-Net: CNN-based Underwater Image Enhancement Using Two Color Space (2021, methodology): “improving the performance of enhancement and restoration by predicted the depth map or transmission map of the underwater image [29]–[31]; employed multiple generators to inference the improved image [32]–[34].”
• Enhancing Underwater Imagery Using Generative Adversarial Networks (2018, background): “Most similar to the work proposed in this paper is the recently proposed WaterGAN [20], which uses an adversarial approach towards generating realistic underwater images.”
• An Underwater Image Enhancement Benchmark Dataset and Beyond (2019, methodology): “[55] proposed a deep learning-based underwater image enhancement model, called WaterGAN.”
• U-Shape Transformer for Underwater Image Enhancement (2021, methodology): “We have collected 8018 underwater images, which are sourced from Bubble Vision1, BIOS2 and existing datasets [1, 9, 26, 31, 32].”
• Real-World Underwater Enhancement: Challenges, Benchmarks, and Solutions Under Natural Light (2019, methodology): “Recently, data-driven CNNs comprising the information from human labels for non-reference quality assessment of natural images have rapidly developed and achieved remarkable performance [51], [52], [53].”