Infrared and Visible Image Fusion with ResNet and zero-phase component analysis

By Hui Li, Xiaojun Wu. Infrared physics & technology, 2018

322 citations recorded by the source corpus.

Abstract In image fusion approaches, feature extraction and processing are key tasks, and the fusion performance is directly affected by the different features and processing methods undertaken. However, most of deep learning-based methods use deep features directly without them. This leads to the fusion performance degradation in some cases. To solve these drawbacks, in our paper, a deep features and zero-phase component analysis (ZCA) based novel fusion framework is proposed. Firstly, the residual network (ResNet) is used to extract deep features from source images. Then ZCA and l 1 -norm are utilized to normalize the deep features and obtain initial weight maps. The final weight maps are obtained by employing a soft-max operation in association with the initial weight maps. Finally, the fused image is reconstructed using a weighted-averaging strategy. Compared with the existing fusion methods, experimental results demonstrate that the proposed framework achieves better performance in both objective assessment and visual quality. The code of our fusion algorithm is available at https://github.com/hli1221/imagefusion_resnet50 .

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/1cb7355e052076ea7c5e5fcd06f8fd783ebcff7a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

MDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion (2018, methodology): “They include: cross bilateral filter method(CBF) [14], discrete cosine harmonic wavelet transform method(DCHWT) [13], joint sparse representation-based method(JSR) [55], the JSR model with saliency detection method(JSRSD) [23], the gradient transfer fusion method(GTF) [30], weighted least square opt”
LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images (2023, background): “These deep learning based fusion approaches can be divided into three categories: (1) Pre-trained deep network based fusion methods [17]; (2) Auto-encoder based fusion [18]; (3) End-to-end learning fusion networks [19]”
NestFuse: An Infrared and Visible Image Fusion Architecture Based on Nest Connection and Spatial/Channel Attention Models (2020, background): “With the rise of deep learning in recent years, a lot of deep learning based methods have been proposed for the image fusion task.”
RFN-Nest: An end-to-end residual fusion network for infrared and visible images (2021, methodology): “In the feature extraction direction, deep learning meth-ods are utilized to extract deep representation of the information conveyed by the source images [22][23][24][25][26].”
CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach (2024, methodology): “In this kind of approaches, deep learning models can be injected in each stage, such as convolutional neural networks (CNNs) for feature extraction [7][9][31], a specific light network is trained to replace the handcrafted fusion strategy [28].”
Visible and Infrared Image Fusion Using Deep Learning (2023, methodology): “map [43] or multiple weight maps [57], [84] based on extracted features, and then use a weighted average strategy to obtain the fused image.”