Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics

By Claire Thomas, Thierry Ranchin, Lucien Wald, Jocelyn Chanussot. IEEE Transactions on Geoscience and Remote Sensing, 2008

659 citations recorded by the source corpus.

Our framework is the synthesis of multispectral images (MS) at higher spatial resolution, which should be as close as possible to those that would have been acquired by the corresponding sensors if they had this high resolution. This synthesis is performed with the help of a high spatial but low spectral resolution image: the panchromatic (Pan) image. The fusion of the Pan and MS images is classically referred as pan-sharpening. A fused product reaches good quality only if the characteristics and differences between input images are taken into account. Dissimilarities existing between these two data sets originate from two causes-different times and different spectral bands of acquisition. Remote sensing physics should be carefully considered while designing the fusion process. Because of the complexity of physics and the large number of unknowns, authors are led to make assumptions to drive their development. Weaknesses and strengths of each reported method are raised and confronted to these physical constraints. The conclusion of this critical survey of literature is that the choice in the assumptions for the development of a method is crucial, with the risk to drastically weaken fusion performance. It is also shown that the Amelioration de la Resolution Spatiale par Injection de Structures concept prevents from introducing spectral distortion into fused products and offers a reliable framework for further developments.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/75f86e5acf1733196038ef973f65ddaa3ff78a15

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

A review of remote sensing image fusion methods (2016, background): “The main emphasis in [19] is that physics must be taken into account and discuss ways to do so.”
A New Pan-Sharpening Method With Deep Neural Networks (2015, methodology): “2376034 preprocessing step in many remote sensing applications such as feature detection and land-cover classification [1].”
Spatiotemporal Reflectance Fusion via Sparse Representation (2012, background): “While a broad range of image fusion methods has been developed in an attempt to combine remotely sensed data from various sensors at different resolutions [7]–[12], most of them have focused on the fusion of panchromatic and multispectral images that combine the HSR of the panchromatic band with the”
A New Pan-Sharpening Method Using a Compressed Sensing Technique (2011, background): “The fusing process, classically referred as pan-sharpening technique, has become a key preprocessing step in many remote sensing applications [2].”
A survey of classical methods and new trends in pansharpening of multispectral images (2011, methodology): “Note, however, that even with geometrically registered PAN and MS images, differences might appear between images as described in [10].”
A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement (2011, methodology): “Moreover, as is well known from [33], the PAN and MS images may present some local instability or dissimilarity, such as object occultation or contrast inversion.”
A Critical Comparison Among Pansharpening Algorithms (2015, background): “Detailed surveys of pansharpening algorithms can be found in [1] and [5]–[7].”
Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening (2021, neutral): “many recently published review articles [5], [6].”
Remote Sensing Image Fusion Based on Two-Stream Fusion Network (2017, background): “Many research e ﬀ orts have been devoted to developing pan-sharpening algorithms during the last decades [1, 2, 3].”
A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods (2021, methodology): “COMPONENT SUBSTITUTION The methods belonging to the CS class rely upon the projection of the MS image into a new space, where the spatial structure is well separated from the spectral information [112].”
Hyperspectral Remote Sensing Data Analysis and Future Challenges (2013, methodology): “All described methods for enhancement of the spatial resolution will generate images at a higher spatial sampling that show higher contrast and finer details, but this does not necessarily guarantee an improvement of the actual spatial resolution [37].”
Hyperspectral Pansharpening: A Review (2015, methodology): “A taxonomy of pansharpening methods can be found in the literature [4], [5], [6].”
Multimodal Classification of Remote Sensing Images: A Review and Future Directions (2015, background): “A wide variety of image fusion methods have been proposed in the remote sensing literature [7], [17], [18], [38], [39], being pan-sharpening methods [24], [32], [35], [40] among the most widespread.”