MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery

By Bruno Aiazzi, Luciano Alparone, Stefano Baronti, Andrea Garzelli, Massimo Selva. 2006

891 citations recorded by the source corpus.

This work presents a multiresolution framework for merging a multispectral image having an arbitrary number of bands with a higher-resolution panchromatic observation. The fusion method relies on the generalized Laplacian pyramid (GLP), which is a multiscale, oversampled structure. The goal is to selectively perform injection of spatial frequencies from an image to another with the constraint of thoroughly retaining the spectral information of the coarser data. The novel idea is that a model of the modulation transfer functions (MTF) of the multispectral scanner is exploited to design the GLP reduction filter. Thus, the interband structure model (IBSM), which is calculated at the coarser scale, where both MS and PAN data are available, can be extended to the finer scale, without the drawback of the poor enhancement occurring when MTFs are assumed to be ideal filters. Experiments carried out on QuickBird data demonstrate that a superior spatial enhancement, besides the spectral quality typical of injection methods, is achieved by means of the MTF-adjusted fusion.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/e77565e73b53d3fe0ed4849b91c8f7cbae1f77d7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Assessment of image fusion procedures using entropy, image quality, and multispectral classification (2008, methodology): “Of the top three algorithms the first two make use of multi-resolution analysis [17-19] while the third [20] used a genetic algorithm to minimize global distortion.”
Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network (2017, methodology): “The performances of DRPNN are compared with five algorithms from different branches for comparisons with the proposed network, including Component Substitution: GS[1], Detail Injection: MTF-GLP[3], SFIM[4], regularization constraint models: ISTS[5] based on total variation and TSSC[6] based on spars”
Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening (2021, methodology): “…(EXP ) [17], the GS sharpening approach [12], the SFIM [13], the PRACS approach [11], the BDSD method [9], the robust BDSD approach (BDSD-PC) [10], the GLP with MTF-matched ﬁlter [58] and multiplicative injection model [59] [GLP-high-pass modulation (HPM)], the GLP with MTF-matched ﬁlter [58]”
SelfS2: Self-Supervised Transfer Learning for Sentinel-2 Multispectral Image Super-Resolution (2023, methodology): “In [12], existing component substitution [13], [14], [15] and multiresolution analysis [16], [17], [18], [19], [20] methods designed for pansharpening are extended to super-resolve bands at 20 m of S2 images via integrating four fine bands into a single band.”
Pansharpening by Convolutional Neural Networks (2016, methodology): “Other methods following the same philosophy work in a MRA setting, e.g., ATWT [15,19–21] or with LP representations [16,22–24].”
Multisource and Multitemporal Data Fusion in Remote Sensing: A Comprehensive Review of the State of the Art (2019, methodology): “[56] B.”
A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening (2017, methodology): “Examples of this type of method are the fusion methods based on wavelet transform [6] or curvelet transform [7], the analysis of modulation transfer function (MTF) [8], [9], and the smoothing filter based intensity modulation (SFIM) method [10].”
Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data (2007, methodology): “lower sharpness and spatial enhancement, which is possibly due to improper digital filtering of the Pan image [17] or”
Target-Adaptive CNN-Based Pansharpening (2017, methodology): “To match the sensor properties, in [16] it is proposed to use an approximation of the sensor MTF.”
Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening (2014, methodology): “MTF method, was first proposed in [4] and successively efficiently implemented through the HPM details injection scheme (9) in [6].”
Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest (2007, methodology): “MRA is adjusted to the MTFs of the MS scanner (one for each band), either directly, as for GLP-CBD [20], or indirectly, as for AWLP, which utilizes Starck and Murtagh Gaussian-like cubic spline ﬁlter [18], whose frequency response matches the typical shape of an isotropic MTF, with an amplitude…”
Hyperspectral Pansharpening: A Review (2015, methodology): “They will be referred to as MTF-Generalized Laplacian Pyramid (MTF-GLP) [45] and MTF-GLP with High Pass Modulation (MTF-GLP-HPM) [42], respectively.”
A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods (2021, methodology): “Indeed, even if nonorthogonality could compromise the spectral quality of fused products [129], its beneficial characteristics, such as the shift-invariance property [130] and the capability of being easily matched to the sensor MTF [47], produce accurate pansharpened images.”
Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images (2008, methodology): “We have referred to the results of the 2006 IEEE Data Fusion Contest [25]–[27], thus comparing to the two algorithms CBD [9] and proportional additive wavelet to the L component (AWLP) [28] which have been awarded as the two most distinguished algorithms [25].”