Fader Networks: Manipulating Images by Sliding Attributes

By Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, Marc'Aurelio Ranzato. 2017

565 citations recorded by the source corpus.

This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space. As a result, after training, our model can generate different realistic versions of an input image by varying the attribute values. By using continuous attribute values, we can choose how much a specific attribute is perceivable in the generated image. This property could allow for applications where users can modify an image using sliding knobs, like faders on a mixing console, to change the facial expression of a portrait, or to update the color of some objects. Compared to the state-of-the-art which mostly relies on training adversarial networks in pixel space by altering attribute values at train time, our approach results in much simpler training schemes and nicely scales to multiple attributes. We present evidence that our model can significantly change the perceived value of the attributes while preserving the naturalness of images.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “…image datasets [22, 31, 13, 69, 37, 24, 41, 68, 15, 27, 46, 8, 39] have been proposed and fostered research and development for automated face detection [35, 21], alignment [66, 44], recognition [57, 49], generation [67, 5, 26, 58], modiﬁcation [3, 32, 19], and attribute classiﬁcation [31, 37].”
Generative Adversarial Networks (2021, neutral): “It was proved successful in many domains such as computer vision [14, 29, 25, 38], semantic segmentation [39, 27, 70, 24], time-series synthesis [9, 23], image editing [61, 36, 19, 3, 75], natural language processing [15, 28, 22], text-to-image generation [59, 58, 54], and many more.”
Unsupervised Machine Translation Using Monolingual Corpora Only (2017, methodology): “Our approach is also reminiscent of the Fader Networks architecture (Lample et al., 2017), where a discriminator is used to remove the information related to specific attributes from the latent states of an autoencoder of images.”
Controllable Abstractive Summarization (2017, background): “Research in computer vision includes style transfer (Gatys et al., 2015) or controllable image generation (Lample et al., 2017).”
Compositional Fairness Constraints for Graph Embeddings (2019, background): “This approach builds upon the recent successes of generative adversarial networks [9], adversarial fairness [15], and adversarial disentanglement [14, 16] in computer vision—adapting these techniques to the domain of graph representation learning.”
Invariant Representations without Adversarial Training (2018, methodology): “We demonstrate a form of unsupervised image manipulation inspired by Fader Networks [14] on the MNIST dataset.”
Recent Advances in Autoencoder-Based Representation Learning (2018, methodology): “rs such as disentanglement (e.g., -VAE [2]) were only applied to synthetic data sets and very structured real data sets at low resolution. In contrast, fully supervised methods, such as FaderNetworks [12], provide representations which capture subtle properties of the data, can be scaled to high-res”
Interpreting the Latent Space of GANs for Semantic Face Editing (2019, methodology): “To achieve this goal, current methods required carefully designed loss functions [26, 10, 32], introduction of additional attribute labels or features [23, 36, 3, 34, 31], or special architectures [11, 30] to train new models.”
Towards Controllable Story Generation (2018, background): “With the recent successes on controllable generation of images (Chen et al., 2016; Siddharth et al., 2017; Lample et al., 2017), dialog responses (Wang et al., 2017), poems (Ghazvininejad et al., 2017), and different styles of text (Hu et al., 2017; Ficler and Goldberg, 2017; Shen et al., 2017; Fu…”
Learning Latent Subspaces in Variational Autoencoders (2018, methodology): “This is the case also with other methods of structuring latent space which have been explored, such as batching data according to labels [12] or use of a discriminator network in a non-generative model [13].”
Disentangling speech from surroundings in a neural audio codec (2022, methodology): “This has been applied to disentanglement of image features [10], sensor anonymization [11], voice conversion [12, 13] and music translation [14].”
Evading the Adversary in Invariant Representation (2018, methodology): “This adversarial solution coincides exactly with the conceptual framework used in a computer vision application for constructing Fader Networks [13].”
Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language (2018, methodology): “With the advance of deep learning and generative models, many techniques have been developed for image editing including the style transfer [2, 4, 5], the drawing based editing [6, 7], and the domain/attribute translation [8–10].”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, background): “Lample et al. proposed in [89] an encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the attribute values directly in the latent space 19 .”
Arbitrary Facial Attribute Editing: Only Change What You Want (2017, methodology): “In comparison with IcGAN [17] and Fader Networks [10], in our AttGAN, no constraint is imposed on the latent representation, which guarantees its representation ability for further editing.”
STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing (2019, methodology): “Following [7, 11], we first evaluate our STGAN for arbitrary facial attribute editing on the CelebA dataset [23] which has been adopted by most relevant works [7, 11, 17, 26].”
A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation (2018, background): “Previous works [25, 15] had discovered that even the disentangled feature to the generator/decoder is binary during training, it can be considered as continuous variable during testing.”