DenseFuse: A Fusion Approach to Infrared and Visible Images

By Hui Li, Xiaojun Wu. IEEE Transactions on Image Processing, 2018

1,849 citations recorded by the source corpus.

In this paper, we present a novel deep learning architecture for infrared and visible images fusion problems. In contrast to conventional convolutional networks, our encoding network is combined with convolutional layers, a fusion layer, and dense block in which the output of each layer is connected to every other layer. We attempt to use this architecture to get more useful features from source images in the encoding process, and two fusion layers (fusion strategies) are designed to fuse these features. Finally, the fused image is reconstructed by a decoder. Compared with existing fusion methods, the proposed fusion method achieves the state-of-the-art performance in objective and subjective assessment.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/ae7327df9b7faaec39a5c20dfdfb9908551c1e71

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jiayi Ma
2. Luc Van Gool
3. Hui Li
4. Radu Timofte
5. Yiannis Demiris
6. Yulun Zhang
7. Xingchen Zhang
8. Xiaojun Wu

In Media Technology

1. Jiayi Ma
2. Hui Li
3. Yiannis Demiris
4. Xingchen Zhang
5. Radu Timofte
6. Luc Van Gool
7. Linfeng Tang
8. Xiaojun Wu

Citations in the corpus

• MDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion (2018, methodology): “commonly used for image fusion are multi-scale transforms [4], [44] and representation learning based methods [3], [19].”
• Visible and Infrared Image Fusion Using Deep Learning (2023, methodology): “A well-known and pioneering AE-based VIF method is DenseFuse [42], which uses MS-COCO [138] to pretrain the AE and different fusion strategies (addition and l 1 -norm) to perform feature fusion.”
• Infrared and Visible Image Fusion with ResNet and zero-phase component analysis (2018, background): “[17] proposed a novel fusion network based on DeepFuse and densenet[18] which is named DenseFuse.”
• YDTR: Infrared and Visible Image Fusion via Y-Shape Dynamic Transformer (2023, methodology): “Fromtoptobottom:infraredimage,visible image,thefusionresultsofRP[8],CVT[9],DenseFuse[18],FusionGAN[19],U2Fusion[20],GANMcC[24],RFN-Nest[23],CSF[29],PPTFusion[32],and theproposedYDTR.”
• DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion (2020, methodology): “From top to bottom: infrared image, visible image, fusion results of DDCTPCA [14], HMSD [47], FPDE [48], GTF [17], DRTV [49], DenseFuse [29], FusionGAN [21] and our DDcGAN.”
• FusionDN: A Unified Densely Connected Network for Image Fusion (2020, methodology): “In (Li and Wu 2018), the encoding network and the decoder are designed to extract and fuse features for the infrared and visible image fusion and the method is also applied to multi-focus image fusion.”
• Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection (2022, methodology): “Early works plugged deep networks into the IVIF process acting as a module of feature extraction or weights generation [8,12, 13,16].”
• CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach (2024, methodology): “Multi-stage image fusion methods divide the fusion process into several stages [20][28].”
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion (2022, methodology): “In the era of DL, CNN-based models for MMIF can be categorized into four main classes: generative adversarial network (GAN)-based models [39,42,43], AE-based models [28, 29, 37, 82], unified models [26, 70, 71, 83, 85] and algorithm unrolling models [11,15,77,89].”
• DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion (2020, methodology): “…section is to study the performance of our proposed model and compare it with other SOTA models, including FusionGAN [Ma et al. , 2019], Densefuse [Li and Wu, 2018], ImageFuse [Li et al. , 2018], DeepFuse [Prab-hakar et al. , 2017], TSIFVS [Bavirisetti and Dhuli, 2016], TVADMM [Guo et al. ,…”
• U2Fusion: A Unified Unsupervised Image Fusion Network (2020, methodology): “We compare U2Fusion with five state-of-the-art methods: HMSD [42], GTF [19], DenseFuse [17], FusionGAN [5], and DDcGAN [24].”
• LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images (2023, methodology): “fusion network (DenseFuse) [22]; a GAN based fusion algorithm (FusionGAN) [32]; a novel end-to-end fusion method (IFCNN) [42]; an ISTA based fusion a network (CUNet) [31]; a”
• Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, positive): “FastSCNN [18] adopts the skip connection in the deep convolutional neural network and proposes a shallow learning module to downsam-ple for fast and efficient multi-branch low-level feature extraction.”
• GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion (2021, positive): “Li and Wu [17] cleverly used the autoencoder structure with dense blocks, in which two traditional fusion strategies are adopted to fuse features in the fusion layer.”
• SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness (2022, background): “DenseFuse [16] is the pioneer of AE-based fusion approaches, which introduces dense blocks in the encoder to achieve effective feature extraction and information utilization.”
• LLVIP: A Visible-infrared Paired Dataset for Low-light Vision (2021, methodology): “Hui Li and Xiao-Jun Wu proposed DenseFuse [10], which incorporated the dense block in the encoder, that is, the outputs of each convoluted layer were connected to each other.”