Infrared and visible image fusion based on target-enhanced multiscale transform decomposition

By Jun Chen, Xuejiao Li, Linbo Luo, Xiaoguang Mei, Jiayi Ma. Information Sciences, 2020

389 citations recorded by the source corpus.

Abstract In this study, we propose a target-enhanced multiscale transform (MST) decomposition model for infrared and visible image fusion to simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images. The Laplacian pyramid is initially used to separately decompose two pre-registered source images into low- and high-frequency bands. The common “max-absolute” fusion rule is performed for fusion for high-frequency bands. We use the decomposed infrared low-frequency information to determine the fusion weight of low-frequency bands and highlight the target. Meanwhile, a regularization parameter is introduced to dominate the proportion of the infrared features in a gentle manner, which can be further adjusted according to user requirements. Finally, we use inverse transform with the Laplacian pyramid (LP) to reconstruct the fused image. Qualitative and quantitative experimental results on publicly available datasets demonstrate that the proposed method can generate fused images with clearly highlighted targets and abundant details. These images exhibit better visual effects and objective metric values than those of five other commonly used MST decomposition methods.

Subfields: Aerospace Engineering, Media Technology

View Paper: https://www.semanticscholar.org/paper/b5914d6f46434ff71368b7248188876e5473442b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Jiayi Ma
2. Guobao Xiao
3. Linfeng Tang
4. Hui Li
5. Josef Kittler
6. Jinde Cao
7. Hao Zhang
8. Shishuang Yu

In Media Technology

1. Hui Li
2. Jiayi Ma
3. Linfeng Tang
4. Xiaojun Wu
5. Josef Kittler
6. Yong Ma
7. Jun Huang
8. Yiannis Demiris

Citations in the corpus

• SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer (2022, background): “On the one hand, the traditional frameworks usually implement complementary information aggregation in the spatial domain [17] or transform domain [19], [27], but neither of them can exchange information between non-adjacent pixels.”
• MDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion (2018, methodology): “commonly used for image fusion are multi-scale transforms [4], [44] and representation learning based methods [3], [19].”
• STDFusionNet: An Infrared and Visible Image Fusion Network Based on Salient Target Detection (2021, methodology): “…or transform domain, which can be divided into five categories according to their principles, including multiscale transform-based methods [2], [17], saliency-based methods [4], [18], sparse representation-based methods [5], [19], optimization-based methods [6], [20], and hybrid meth-ods [7],…”
• Infrared and Visible Image Fusion via Decoupling Network (2022, methodology): “These methods mainly rely on predesigned feature extraction criteria and feature fusion criteria, which commonly include multiscale-based methods [23], [24], edge-preserving filter-based methods [25], and total variation-based methods [26].”
• SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness (2022, background): “More specifically, traditional fusion frameworks involve broadly five categories, i.e. , multi-scale transformation (MST)- based [8], [9], [10], sparsity representation (SR)-based [11], subspace-based [12], saliency-based [13], and optimization-based [14], [15] fusion methods.”