DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs
By K. Ram Prabhakar, V. Sai Srikar, R. Venkatesh Babu. IEEE International Conference on Computer Vision, 2017
672 citations recorded by the source corpus.
We present a novel deep learning architecture for fusing static multi-exposure images. Current multi-exposure fusion (MEF) approaches use hand-crafted features to fuse input sequence. However, the weak hand-crafted representations are not robust to varying input conditions. Moreover, they perform poorly for extreme exposure image pairs. Thus, it is highly desirable to have a method that is robust to varying input conditions and capable of handling extreme exposure without artifacts. Deep representations have known to be robust to input conditions and have shown phenomenal performance in a supervised setting. However, the stumbling block in using deep learning for MEF was the lack of sufficient training data and an oracle to provide the ground-truth for supervision. To address the above issues, we have gathered a large dataset of multi-exposure image stacks for training and to circumvent the need for ground truth images, we propose an unsupervised deep learning framework for MEF utilizing a no-reference quality metric as loss function. The proposed approach uses a novel CNN architecture trained to learn the fusion operation without reference ground truth image. The model fuses a set of common low level features extracted from each image to generate artifact-free perceptually pleasing results. We perform extensive quantitative and qualitative evaluation and show that the proposed technique outperforms existing state-of-the-art approaches for a variety of natural images.
Subfields: Computer Vision and Pattern Recognition, Media Technology
View Paper: https://www.semanticscholar.org/paper/3c59007529da04b8515de9bc1c326a3ce82c8e9e
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
In Media Technology
Citations in the corpus
- SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image Fusion (2021, methodology): “7, the lamps are blurred and their edges are even invisible in the results of other three methods except DeepFuse, U2Fusion and our SDNet.”
- DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion (2020, background): “[28] proposed an unsupervised deep learning framework for multi-exposure fusion.”
- NestFuse: An Infrared and Visible Image Fusion Architecture Based on Nest Connection and Spatial/Channel Attention Models (2020, methodology): “We choose twelve typical and state-of-the-art fusion methods to evaluate the fusion performance, including: cross bilateral filter fusion method (CBF) [39], discrete cosine harmonic wavelet transform fusion method (DCHWT) [40], joint SR based fusion method (JSR) [41], the joint sparse representation”
- Artificial intelligence in the creative industries: a review (2020, background): “all-in-focus image is created using multiple images of the same scene taken with dierent focal settings [189] (Fig. 10 (a)). Multi-exposure deep fusion is used to create high-dynamic range images in [190]. A review of deep learning for pixel-level image fusion can be found in [288]. Recently, GANs h”
- GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion (2021, methodology): “In multiexposure image fusion, a no-reference quality metric is used as the loss function to train an unsupervised network, which fuses a set of common low-level features extracted from each image to generate promising results [31].”
- MDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion (2018, methodology): “They include: cross bilateral filter method(CBF) [14], discrete cosine harmonic wavelet transform method(DCHWT) [13], joint sparse representation-based method(JSR) [55], the JSR model with saliency detection method(JSRSD) [23], the gradient transfer fusion method(GTF) [30], weighted least square opt”
- FusionDN: A Unified Densely Connected Network for Image Fusion (2020, methodology): “…a CNN for projection, mapping, and reconstruction to solve the pansharpening problem. ii) For digital photography images, for the first time, (Prabhakar, Srikar, and Babu 2017) introduced deep learning in multi-exposure image fusion by building a novel CNN and applying MEF-SSIM to realize…”
- U2Fusion: A Unified Unsupervised Image Fusion Network (2020, methodology): “…illustration of different image fusion tasks (first row: source images, second row (from left to right): fusion results of FusionGAN [5], U2Fusion, NSCT [6], U2Fusion, Deepfuse [7], U2Fusion, GFDF [8] and U2Fusion). cates the similarity relationship between the source images and the fusion result.”
- Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, positive): “For instance, EDANet [17] proposes a new network architecture with efficient dense modules using asymmetric convolution.”
- Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and Intensity (2020, background): “In multi-exposure image fusion, Prabhakar et al. (Ram Prabhakar, Sai Srikar, and Venkatesh Babu 2017) proposed an unsupervised deep learning framework that utilizes a no-reference quality metric MEF-SSIM (Ma, Zeng, and Wang 2015) as the loss function and can produce satisfactory fused results.”
- RFN-Nest: An end-to-end residual fusion network for infrared and visible images (2021, methodology): “Although many methods [22][24][34][25] now use deep features to achieve good performance, the heuristic approach to selecting a suitable fusion strategy is their weakness.”
- DenseFuse: A Fusion Approach to Infrared and Visible Images (2018, methodology): “In traditional CNN based network, with the increase of network depth, a degradation problem [15] has been exposed and many information which are extracted by middle layers are not used thoroughly.”
- SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer (2022, background): “Subsequently, Zhang et al. [20] designed the first convolution neural network for general image fusion with reference to DeepFuse [13].”