Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion

By Jiayi Ma, Wei Yu, Chen Chen, Pengwei Liang, Xiaojie Guo, Junjun Jiang. Information Fusion, 2020

446 citations recorded by the source corpus.

Abstract Pan-sharpening in remote sensing image fusion refers to obtaining multi-spectral images of high-resolution by fusing panchromatic images and multi-spectral images of low-resolution. Recently, convolution neural network (CNN)-based pan-sharpening methods have achieved the state-of-the-art performance. Even though, two problems still remain. On the one hand, the existing CNN-based strategies require supervision, where the low-resolution multi-spectral image is obtained by simply blurring and down-sampling the high-resolution one. On the other hand, they typically ignore rich spatial information of panchromatic images. To address these issues, we propose a novel unsupervised framework for pan-sharpening based on a generative adversarial network, termed as Pan-GAN, which does not rely on the so-called ground-truth during network training. In our method, the generator separately establishes the adversarial games with the spectral discriminator and the spatial discriminator, so as to preserve the rich spectral information of multi-spectral images and the spatial information of panchromatic images. Extensive experiments are conducted to demonstrate the effectiveness of the proposed Pan-GAN compared with other state-of-the-art pan-sharpening approaches. Our Pan-GAN has shown promising performance in terms of qualitative visual effects and quantitative evaluation metrics.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/f1828485cb73fe71607f1ab6ea6bd9bee869f8bf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jocelyn Chanussot
2. Jiayi Ma
3. Linfeng Tang
4. Gemine Vivone
5. Giuseppe Scarpa
6. Yong Ma
7. Hao Zhang
8. Jun Huang

In Media Technology

1. Jocelyn Chanussot
2. Jiayi Ma
3. Linfeng Tang
4. Giuseppe Scarpa
5. Gemine Vivone
6. Hao Zhang
7. Yong Ma
8. Jun Huang

Citations in the corpus

• SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image Fusion (2021, methodology): “Although using intensity loss and gradient loss (Ma et al. 2019, 2020) is a common practice in a specific image fusion task (Szeliski et al. 2011), it is non-trivial to extend them to other image fusion tasks.”
• Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review (2022, methodology): “com/zhysora/UCGAN Pan-GAN (Ma et al., 2020) TensorFlow https://github.”
• SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer (2022, methodology): “Then, they applied GAN to a range of fusion tasks, such as infrared and visible image fusion [55], multi-exposure image fusion [25], multi-focus image fusion [56], and pan-sharpening [57].”
• A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods (2021, background): “Another interesting example is the Pan-GAN model [109], which relies on spectral–spatial multiobjective adversarial training using two dedicated discriminators.”
• GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion (2021, methodology): “Analogously, Ma et al. [34] adopted GAN with dual discriminators to transform pansharpening into a multitask learning problem, saying spectrum preservation and spatial preservation.”