DoveNet: Deep Image Harmonization via Domain Verification

By Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, Weiyu Li, Liqing Zhang. 2019

261 citations recorded by the source corpus.

Image composition is an important operation in image processing, but the inconsistency between foreground and background significantly degrades the quality of composite image. Image harmonization, aiming to make the foreground compatible with the background, is a promising yet challenging task. However, the lack of high-quality publicly available dataset for image harmonization greatly hinders the development of image harmonization techniques. In this work, we contribute an image harmonization dataset iHarmony4 by generating synthesized composite images based on COCO (resp., Adobe5k, Flickr, day2night) dataset, leading to our HCOCO (resp., HAdobe5k, HFlickr, Hday2night) sub-dataset. Moreover, we propose a new deep image harmonization method DoveNet using a novel domain verification discriminator, with the insight that the foreground needs to be translated to the same domain as background. Extensive experiments on our constructed dataset demonstrate the effectiveness of our proposed method. Our dataset and code are available at https://github.com/bcmi/Image_Harmonization_Datasets.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Ming-Hsuan Yang
2. Luc Van Gool
3. Yujiu Yang
4. Weihao Xia
5. Dacheng Tao
6. Jing-Hao Xue
7. Zuxuan Wu
8. Xiaogang Wang

Citations in the corpus

• FocalClick: Towards Practical Interactive Image Segmentation (2022, methodology): “We also report the performance of our model trained on SBD [15], and a large combined dataset [2, 7, 13, 24, 26, 39, 45, 48].”
• DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (2022, background): “[13] Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, and Liqing Zhang.”
• AnyDoor: Zero-shot Object-level Image Customization (2023, unknown): “The training supervision is a mean square error loss as:”
• Towards Open-World Text-Guided Face Image Generation and Manipulation (2021, methodology): “Different backgrounds can be further added using image composition or harmonization methods like DoveNet [61].”
• TediGAN: Text-Guided Diverse Image Generation and Manipulation (2020, methodology): “Different backgrounds can be further added using image composition or harmonization methods like DoveNet [7].”
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection (2021, methodology): “Finally, we use the image harmonization method in [6] for post-processing.”