Learning Residual Images for Face Attribute Manipulation

By Wei Shen, Rujie Liu. 2016

264 citations recorded by the source corpus.

Face attributes are interesting due to their detailed description of human faces. Unlike prior researches working on attribute prediction, we address an inverse and more challenging problem called face attribute manipulation which aims at modifying a face image according to a given attribute value. Instead of manipulating the whole image, we propose to learn the corresponding residual image defined as the difference between images before and after the manipulation. In this way, the manipulation can be operated efficiently with modest pixel modification. The framework of our approach is based on the Generative Adversarial Network. It consists of two image transformation networks and a discriminative network. The transformation networks are responsible for the attribute manipulation and its dual operation and the discriminative network is used to distinguish the generated images from real images. We also apply dual learning to allow transformation networks to learn from each other. Experiments show that residual images can be effectively learned and used for attribute manipulations. The generated images remain most of the details in attribute-irrelevant areas.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation (2017, background): “Generative adversarial networks (GANs) [3] have shown remarkable results in various computer vision tasks such as image generation [1, 6, 23, 31], image translation [7, 8, 32], super-resolution imaging [13], and face image synthesis [9, 15, 25, 30].”
Achieving Human Parity on Automatic Chinese to English News Translation (2018, background): “Ever since dual learning was proposed, it has been successfully applied to various real-world problems such as question answering [35], image classification [47], image segmentation [25], image to image translation [50, 52, 24], face attribute manipulation [31], and machine translation [19, 43, 23, ”
Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis (2017, background): “Recently, modiﬁed GAN architectures, conditional GAN [21] in particular, have been successfully applied to vision tasks like image inpainting [24], super-resolution [19], style transfer [20], face attribute manipulation [29] and even data augmentation for boosting classiﬁcation models [30,43].”
Arbitrary Facial Attribute Editing: Only Change What You Want (2017, methodology): “Recently, considerable progress has been made on facial attribute editing [11, 17, 12, 20, 23, 10].”
An Introduction to Image Synthesis with Generative Adversarial Nets (2018, background): “al [88] propose to learn residual images of attributes.”
Deep Identity-aware Transfer of Facial Attributes (2016, methodology): “pensive and cannot obtain high quality results. Perarnau et al. [17] adopt an encoder-decoder architecture, where attribute transfer can be conducted by editing the latent representation. Shen et al. [20] learn the residual image in the GAN framework, and adopt dual learning to learn two reverse att”
GANimation: Anatomically-aware Facial Animation from a Single Image (2018, background): “Most works have tackled the task on attribute editing [17,26,31] trying to modify attribute categories such as adding glasses, changing color hair, gender swapping and aging.”
R³Net: Recurrent Residual Refinement Network for Saliency Detection (2018, background): “, 2016] and face attribute manipulation [Shen and Liu, 2017].”
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals (2019, methodology): “Following the pattern, applications and explorations of GANs have been done for face completion [37], facial attribute manipulation [49, 16, 24], frontal view synthesis [26], facial reenactment [31, 53, 59], identity-preserving synthesis [8] and expression editing [19].”
Disentangled Face Attribute Editing via Instance-Aware Latent Space Search (2021, methodology): “Face attribute editing has attracted much attention in recent years and numerous algorithms have been proposed [Shen and Liu, 2017; Choi et al. , 2018; Zhang et al. , 2018; Bahng et al. , 2020; Awiszus et al. , 2019; Gu et al. , 2019].”
ACE: Adapting to Changing Environments for Semantic Segmentation (2019, unknown): “To control the generation process, a multitude of additional information has been incorporated including labels [36], text [41], attributes [49], and images [21, 25].”
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset (2022, unknown): “Due to CelebV-HQ can be considered as a video version of CelebA-HQ [42] which is a commonly used dataset and its facial attributes annotation is successful in many works [69,76,93,28,17,44,45,13].”
DeepFake Detection by Analyzing Convolutional Traces (2020, methodology): “The authors used CELEBA [27] and LFW [16] datasets, and performed various tests comparing ATTGAN with VAE/GAN [ 23], Ic-GAN [30] and STARGAN [6], Fader Networks [21], Shen et al. [37] and CycleGAN [45].”
A survey on face data augmentation for the training of deep neural networks (2019, methodology): “Examples of glasses wearing (the left three columns) and removal (the right three columns) [32].”
DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation (2018, background): “Various prior information, including labels [38], text [39], attributes [40], images [41,42] has been explored to condition the generation process.”
VITON: An Image-Based Virtual Try-on Network (2017, background): “To incorporate desired properties in generated samples, researchers also utilize different signals, in the form of class labels [33], text [37], attributes [41], etc.”
STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing (2019, methodology): “While several methods have been proposed to learn single translation model for each specific attribute editing task [5, 20, 29, 32], they suffer from the limitation of image-to-image translation and cannot well scale to arbitrary attribute editing.”