Faceshop

By Tiziano Portenier, Qiyang Hu, Attila Szabó, Siavash Arjomand Bigdeli, Paolo Favaro, Matthias Zwicker. 2018

98 citations recorded by the source corpus.

We present a novel system for sketch-based face image editing, enabling users to edit images intuitively by sketching a few strokes on a region of interest. Our interface features tools to express a desired image manipulation by providing both geometry and color constraints as user-drawn strokes. As an alternative to the direct user input, our proposed system naturally supports a copy-paste mode, which allows users to edit a given image region by using parts of another exemplar image without the need of hand-drawn sketching at all. The proposed interface runs in real-time and facilitates an interactive and iterative workflow to quickly express the intended edits. Our system is based on a novel sketch domain and a convolutional neural network trained end-to-end to automatically learn to render image regions corresponding to the input strokes. To achieve high quality and semantically consistent results we train our neural network on two simultaneous tasks, namely image completion and image translation. To the best of our knowledge, we are the first to combine these two tasks in a unified framework for interactive image editing. Our results show that the proposed sketch domain, network architecture, and training procedure generalize well to real user input and enable high quality synthesis results without additional post-processing.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Mask-Guided Portrait Editing With Conditional GANs (2019, background): “Deep Visual Manipulation Image editing has benefited a lot from the rapid growth of deep neural networks, including image completion [23], super-resolution [28], deep analogy [30], and sketch-based portrait editing [39, 34], to name a few.”
SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations (2021, background): “…to directly generate an image based on a user input, and have demonstrated success on a variety of tasks including image synthesis and editing (Portenier et al., 2018; Chen & Koltun, 2017; Dekel et al., 2018; Wang et al., 2018; Park et al., 2019; Zhu et al., 2020b; Jo & Park, 2019; Liu et…”
Semantic photo manipulation with a generative image prior (2019, background): “Recent work further integrates user interaction into end-to-end learning systems, enabling interactive applications such as user-guided colorization [Zhang et al. 2017b] and sketchbased face editing [Portenier et al. 2018].”
SC-FEGAN: Face Editing Generative Adversarial Network With User’s Sketch and Color (2019, methodology): “In another work, the face editing system FaceShop [12], which accepts sketch and color inputs from the user, was introduced.”
DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation (2021, background): “Conditional GAN methods [8, 14, 25, 38, 40, 59, 68, 69] learn direct mappings from original images to target images.”