Mask-Guided Portrait Editing With Conditional GANs

By Shuyang Gu, Jianmin Bao, Hao Yang, Dong Chen, Fang Wen, Lu Yuan. 2019

151 citations recorded by the source corpus.

Portrait editing is a popular subject in photo manipulation.The Generative Adversarial Network (GAN) advances the generating of realistic faces and allows more face editing. In this paper, we argue about three issues in existing techniques: diversity, quality, and controllability for portrait synthesis and editing. To address these issues, we propose a novel end-to-end learning framework that leverages conditional GANs guided by provided face masks for generating faces. The framework learns feature embeddings for every face component (e.g., mouth, hair, eye), separately, contributing to better correspondences for image translation, and local face editing. With the mask, our network is available to many applications, like face synthesis driven by mask, face Swap+ (including hair in swapping), and local manipulation. It can also boost the performance of face parsing a bit as an option of data augmentation.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

SEAN: Image Synthesis With Semantic Region-Adaptive Normalization (2019, unknown): “Among various image-to-image translation problems, semantic image synthesis is a particularly useful genre as it enables easy user control by modifying the input semantic layout image [29, 5, 16, 39].”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, background): “fication; Pixel-DTGAN [133] for cloth image; Gene-GAN [134] and Gaussian Poisson GAN (GP-GAN) [47] for object transfiguration; Semi-Supervised (SS-GAN) [135] for semantic segmentation; Face-mask-CGAN [136] for portrait editing; Vital [137] and Sample-Level GAN (S-GAN) [138] for object tracking; Semi”
Semantically Multi-Modal Image Synthesis (2020, unknown): “[12] built several auto-encoders for each face component to extract different component representations which are then merged into the next foreground generator in the task of portrait editing.”
Paint by Example: Exemplar-based Image Editing with Diffusion Models (2022, background): “Another popular direction of works [5, 18, 36, 65, 73, 75] utilize semantic mask to control the editing.”
Collaging Class-specific GANs for Semantic Image Synthesis (2021, background): “Others [36, 14, 21, 10, 13, 29, 31] model an image via different components, e.”
GIQA: Generated Image Quality Assessment (2020, background): “Recent studies have shown remarkable success in generative models for their wide applications like high quality image generation [1,2], image-to-image translation [3,4,5,6], data augmentation [7,8], and so on.”
PriorGAN: Real Data Prior for Generative Adversarial Nets (2020, background): “The capability of GANs advances many important and useful applications, like high quality image generation [15, 4], image-to-image translation [14, 33], image editing [23, 9], and so on.”
Disentangled Face Attribute Editing via Instance-Aware Latent Space Search (2021, methodology): “Face attribute editing has attracted much attention in recent years and numerous algorithms have been proposed [Shen and Liu, 2017; Choi et al. , 2018; Zhang et al. , 2018; Bahng et al. , 2020; Awiszus et al. , 2019; Gu et al. , 2019].”
Cross-Domain Correspondence Learning for Exemplar-Based Image Translation (2020, methodology): “In order to consider the mask (or edge) in the training, some methods [10, 46, 5] explicitly separate the exemplar image into semantic regions and learns to synthesize different parts individually.”
Generative Adversarial Networks for Face Generation: A Survey (2022, background): “Image-to-image translation and image-to-video translation [47] are quite common for face image mapping from one style domain to another or from a stable face image to a talking face.”