SC-FEGAN: Face Editing Generative Adversarial Network With User’s Sketch and Color

By Youngjoo Jo, Jongyoul Park. 2019

327 citations recorded by the source corpus.

We present a novel image editing system that generates images as the user provides free-form masks, sketches and color as inputs. Our system consists of an end-to-end trainable convolutional network. In contrast to the existing methods, our system utilizes entirely free-form user input in terms of color and shape. This allows the system to respond to the user’s sketch and color inputs, using them as guidelines to generate an image. In this work, we trained the network with an additional style loss, which made it possible to generate realistic results despite large portions of the image being removed. Our proposed network architecture SC-FEGAN is well suited for generating high-quality synthetic images using intuitive user inputs.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, background): “…and texture, edges and contours, or hand-engineered architectures (Altinel et al., 2018; Ding et al., 2018; Iizuka et al., 2017; Jiang et al., 2019; Jo & Park, 2019; Lahiri et al., 2017; Liu et al., 2019a; Nazeri et al., 2019; Ren et al., 2019; Sagong et al., 2019; Wang et al., 2018; Xie et al.,…”
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, methodology): “Another great idea is to use GANs conditioned on sketches and color information to ll in regions in a face image. is strategy was used by FaceShop (Portenier et al. 2018) and SC-FEGAN (Jo and Park 2019).”
Image2StyleGAN++: How to Edit the Embedded Images? (2019, neutral): “Such embedding algorithms can be used to analyze the limitations of GANs [5], do image inpainting [8, 39, 38, 36], local image editing [40, 17], global image transformations such as image morphing and expression transfer [1], and few-shot video generation [35, 34].”
LOHO: Latent Optimization of Hairstyles via Orthogonalization (2021, background): “Although recent work [16, 23, 21] showed progress on using GANs for hair generation, these methods do not allow for intuitive control over the synthesized hair.”
LaFIn: Generative Landmark Guided Face Inpainting (2019, background): “Besides, [11, 33] directly ask users to manually label edges for generating corresponding results.”
Image Inpainting with Learnable Feature Imputation (2020, methodology): “Furthermore, several studies [17,40,41,47] propose to use Wasserstein GAN [2] with gradient penalties [10]; however, the standard gradient penalty causes training instability for image-inpainting models, as we discuss in Section 3.”
Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content (2020, background): “Beneﬁted from the powerful ability of GAN, it enjoys pervasive applications on tasks such as style transfer [52, 3], image inpainting [45, 15, 47, 48, 27], and image editing [20, 5, 24, 31].”
Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Poorly-Drawn Sketches (2019, methodology): “The calibrated sketches can be directly fed into other existing ”well-drawn” sktech2image methods [10], [11] to produce more diverse and more photo-realistic results.”
DeepPrivacy: A Generative Adversarial Network for Face Anonymization (2019, background): “Image Inpainting is a closely related task to what we are trying to solve, and it is a widely researched area for generative models [10,15,17,29].”
Towards Open-World Text-Guided Face Image Generation and Manipulation (2021, methodology): “, label [20], [21], sketch [22], [23], text [24], [25], or control vector like gaze [26] or light direction [27].”
Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval (2022, background): “Users can retrieve photos [8], create [51] imaginative visual-art, or edit existing photos [22] through repeated interactions with the AI agent.”
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation (2019, background): “Because GAN can generate realistic images, it enjoys pervasive applications on tasks such as image-to-image translation [14, 47, 25, 38, 30], image inpainting [24, 44, 45, 15], and virtual try-on [41, 9, 3, 37].”
Deep Learning for Free-Hand Sketch: A Survey (2020, background): “sketch to photo [267], (b) sketch & photo to photo [22], [263], (c) sketch/edge & color to photo [58].”