GP-GAN: Towards Realistic High-Resolution Image Blending

By Huikai Wu, Shuai Zheng, Junge Zhang, Kaiqi Huang. 2017

282 citations recorded by the source corpus.

It is common but challenging to address high-resolution image blending in the automatic photo editing application. In this paper, we would like to focus on solving the problem of high-resolution image blending, where the composite images are provided. We propose a framework called Gaussian-Poisson Generative Adversarial Network (GP-GAN) to leverage the strengths of the classical gradient-based approach and Generative Adversarial Networks. To the best of our knowledge, it's the first work that explores the capability of GANs in high-resolution image blending task. Concretely, we propose Gaussian-Poisson Equation to formulate the high-resolution image blending problem, which is a joint optimization constrained by the gradient and color information. Inspired by the prior works, we obtain gradient information via applying gradient filters. To generate the color information, we propose a Blending GAN to learn the mapping between the composite images and the well-blended ones. Compared to the alternative methods, our approach can deliver high-resolution, realistic images with fewer bleedings and unpleasant artifacts. Experiments confirm that our approach achieves the state-of-the-art performance on Transient Attributes dataset. A user study on Amazon Mechanical Turk finds that the majority of workers are in favor of the proposed method. The source code is available in \urlhttps://github.com/wuhuikai/GP-GAN, and there's also an online demo in \urlhttp://wuhuikai.me/DeepJS.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Generative Adversarial Networks (2021, neutral): “Researchers are also looking into techniques to reduce the quantity of data necessary for GAN training, also known as semi-supervised learning, or to enrich the Training Data with generative models.”
Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View (2019, background): “face frontal view generation [50], generating new human poses [51], photos to emojis [52], photograph editing [53]–[55], face aging [56], [57], photo blending [58], super resolution [59]–[61], photo inpainting [62]–[64], video prediction [65], and 3D object generation [66], [67]; among”
Style transfer-based image synthesis as an efficient regularization technique in deep learning (2019, background): “convert sketches to images) [24], image blending (mixing selected parts of two images to get a new one) [25], image inpainting (restoring missing pieces of an image) [26] and unconditional [27] and conditional synthesis [28].”
How Generative Adversarial Networks and Their Variants Work (2017, methodology): “Gaussian-Poisson GAN (GP-GAN) [132] suggests a high-resolution image blending framework using GAN and a classic image blending gradient-based approach [31].”
FSGAN: Subject Agnostic Face Swapping and Reenactment (2019, methodology): “Inspired by previous uses of Poisson blending for inpainting [51] and blending [49], we propose a novel Poisson blending loss.”
How Generative Adversarial Networks and Its Variants Work : An Overview of GAN (2017, methodology): “GP-GAN [127] applies a high-resolution image blending framework using GAN and a classic image blending gradient based approach [31].”
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (2022, background): “[70] Huikai Wu, Shuai Zheng, Junge Zhang, and Kaiqi Huang.”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, methodology): “GP-GAN was the first framework that used GAN in the blending of images which have very HR.”
Data augmentation for improving deep learning in image classification problem (2018, background): “of two images to get a new one) [15], image inpainting”
Applications of Generative Adversarial Networks (GANs): An Updated Review (2019, background): “[123] proposed Gaussian-Poisson GAN (GPGAN), a framework that combines the strengths of classical gradient-based approaches and GANs, which is the first work that explores the capability of GANs in high-resolution image blending task as shown in Fig.”
Text as Neural Operator:Image Manipulation by Text Instruction (2020, background): “Numerous image stylization [21, 43] and blending [23, 70] approaches augment the images by referencing an exemplar image.”
Generative Adversarial Networks (GANs) (2020, background): “Generation of realistic images has wide range of practical applications, such as anime character generation [159]–[164], image synthesis [165]–[168], super resolution [10], [124], [169]–[177], image editing and blending [178], [179], inpainting [125], [180]–[182], interactive image generation [183],”
Artificial intelligence in the creative industries: a review (2020, unknown): “in [191]), with an example of image blending using a guided mask in [192].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “GANs have been utilized in other image processing and computer vision tasks [354]–[357] such as object transfiguration [358], [359], semantic segmentation [360], visual saliency prediction [361], object tracking [362], [363], image dehazing [364]–[366], natural image matting [367], image inpainting ”