Infrared and Visible Image Fusion using a Deep Learning Framework

By Hui Li, Xiaojun Wu, Josef Kittler. International Conference on Pattern Recognition, 2018

495 citations recorded by the source corpus.

In recent years, deep learning has become a very active research tool which is used in many image processing fields. In this paper, we propose an effective image fusion method using a deep learning framework to generate a single image which contains all the features from infrared and visible images. First, the source images are decomposed into base parts and detail content. Then the base parts are fused by weighted-averaging. For the detail content, we use a deep learning network to extract multi-layer features. Using these features, we use $l_{1}$-norm and weighted-average strategy to generate several candidates of the fused detail content. Once we get these candidates, the max selection strategy is used to get the final fused detail content. Finally, the fused image will be reconstructed by combining the fused base part and the detail content. The experimental results demonstrate that our proposed method achieves state-of-the-art performance in both objective assessment and visual quality. The Code of our fusion method is available at https://github.com/exceptionLi/imagefusion_deeplearning.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/ec9cb72f3132d770627063ad0e6fcb6bf3ce09a5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Hui Li
2. Xiaojun Wu
3. Josef Kittler
4. Xiao-Jun Wu
5. Tariq Salim Durrani
6. Linfeng Tang
7. Jiayi Ma
8. Jun Huang

In Computer Vision and Pattern Recognition

1. Jiayi Ma
2. Hui Li
3. Xiaojun Wu
4. Linfeng Tang
5. Josef Kittler
6. Xiao-Jun Wu
7. Yong Ma
8. Han Xu

In Media Technology

1. Jiayi Ma
2. Hui Li
3. Xiaojun Wu
4. Linfeng Tang
5. Josef Kittler
6. Xiao-Jun Wu
7. Han Xu
8. Yong Ma

Citations in the corpus

• Infrared and Visible Image Fusion with ResNet and zero-phase component analysis (2018, methodology): “…MI pixel [32], F MI dct [32] and F MI w [32] which calculate mutual information (FMI) for the pixel, discrete cosine and wavelet features, respectively; N abf [33] denotes the rate of noise or artifacts added to the fused image by the fusion process; and modified structural similarity( SSIM a )[17].”
• NestFuse: An Infrared and Visible Image Fusion Architecture Based on Nest Connection and Spatial/Channel Attention Models (2020, methodology): “We still need to choose one λ values for our image fusion task based on the test images.”
• RFN-Nest: An end-to-end residual fusion network for infrared and visible images (2021, methodology): “In the feature extraction direction, deep learning meth-ods are utilized to extract deep representation of the information conveyed by the source images [22][23][24][25][26].”
• MDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion (2018, methodology): “preserved in fused images; the standard deviation (SD) [37]; Qabf [52] reflects the quality of visual information obtained from the fusion of the input images; F M Ip , F M Iw and F M Idct [8] calculate fast mutual information (F M I ), discrete cosine and wavelet features, respectively; SC D [1] ca”
• SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer (2022, methodology): “One form of CNN participating in image fusion employs the pre-trained networks to realize activity level measurement and generate a weight map for hand-crafted features [5], [43].”
• CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach (2024, methodology): “In this kind of approaches, deep learning models can be injected in each stage, such as convolutional neural networks (CNNs) for feature extraction [7][9][31], a specific light network is trained to replace the handcrafted fusion strategy [28].”
• LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images (2023, background): “In the training phase, VGG-16 [38], trained on ImageNet, is utilized to be the loss network and four conv-blocks (their indices are 1,2,3,4) are chosen to extract features, as shown in Fig.”
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “, multi-modal Bayesian fusion [90], multi-level fusion [471], image fusion [472], data fusion [473], etc.”
• AttentionFGAN: Infrared and Visible Image Fusion Using Attention-Based Generative Adversarial Networks (2021, unknown): “decomposed the source image into base parts and detail contents, then the base parts and detail contents were fused by different deep learning based fusion strategies [27].”
• DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion (2020, background): “In [20], Li et al . decomposed the source images into base parts and detail content.”