MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion

By Hao Zhang, Zhuliang Le, Zhenfeng Shao, Han Xu, Jiayi Ma. Information Fusion, 2021

318 citations recorded by the source corpus.

Abstract Multi-focus image fusion is an enhancement method to generate full-clear images, which can address the depth-of-field limitation in imaging of optical lenses. Most existing methods generate the decision map to realize multi-focus image fusion, which usually lead to detail loss due to misclassification, especially near the boundary line of the focused and defocused regions. To overcome this challenge, this paper presents a new generative adversarial network with adaptive and gradient joint constraints to fuse multi-focus images. In our model, an adaptive decision block is introduced to determine whether source pixels are focused or not based on the difference of repeated blur. Under its guidance, a specifically designed content loss can dynamically guide the optimization trend, that is, force the generator to produce a fused result of the same distribution as the focused source images. To further enhance the texture details, we establish an adversarial game so that the gradient map of the fused result approximates the joint gradient map constructed based on the source images. Our model is unsupervised without requiring ground-truth fused images for training. In addition, we release a new dataset containing 120 high-quality multi-focus image pairs for benchmark evaluation. Experimental results demonstrate the superiority of our method over the state-of-the-art in terms of both subjective visual effect and quantitative metrics. Moreover, our method is about one order of magnitude faster compared with the state-of-the-art.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/dc728d8c1d8fb30c7568c7e0e90fbcd266b47cc5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jiayi Ma
2. Hui Li
3. Linfeng Tang
4. Xiaojun Wu
5. Josef Kittler
6. Hao Zhang
7. Yong Ma
8. Jun Huang

In Media Technology

1. Jiayi Ma
2. Hui Li
3. Linfeng Tang
4. Xiaojun Wu
5. Josef Kittler
6. Yiannis Demiris
7. Xingchen Zhang
8. Hao Zhang

Citations in the corpus

• LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images (2023, background): “It is also relevant in many other applications, such as photographic image processing [3], multi-modal medical image generation [4] [5], intelligent surveillance [6], multimodal visual object tracking [7] [8], etc.”
• SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer (2022, methodology): “Then, they applied GAN to a range of fusion tasks, such as infrared and visible image fusion [55], multi-exposure image fusion [25], multi-focus image fusion [56], and pan-sharpening [57].”
• GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion (2021, methodology): “The MFF-GAN proposed by Zhang et al. [30] realizes multifocus image fusion with high detail preservation through a well-designed loss function, which can avoid the information loss near the boundary line that appears in the previous decision map-based method.”