Attribute2Image: Conditional Image Generation from Visual Attributes

By Xinchen Yan, Jimei Yang, Kihyuk Sohn, Honglak Lee. 2015

808 citations recorded by the source corpus.

This paper investigates a novel problem of generating images from visual attributes. We model the image as a composite of foreground and background and develop a layered generative model with disentangled latent variables that can be learned end-to-end using a variational auto-encoder. We experiment with natural images of faces and birds and demonstrate that the proposed models are capable of generating realistic and diverse samples with disentangled latent representations. We use a general energy minimization algorithm for posterior inference of latent variables given novel images. Therefore, the learned generative models show excellent quantitative and visual results in the tasks of attribute-conditioned image reconstruction and completion.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “…image datasets [22, 31, 13, 69, 37, 24, 41, 68, 15, 27, 46, 8, 39] have been proposed and fostered research and development for automated face detection [35, 21], alignment [66, 44], recognition [57, 49], generation [67, 5, 26, 58], modiﬁcation [3, 32, 19], and attribute classiﬁcation [31, 37].”
Generative Adversarial Text to Image Synthesis (2016, background): “…the object category encoded into a vector (Farhadi et al., 2009; Kumar et al., 2009; Parikh & Grauman, 2011; Lampert et al., 2014), in particular to enable zero-shot visual recognition (Fu et al., 2014; Akata et al., 2015), and recently for conditional image generation (Yan et al., 2015).”
Unsupervised Image-to-Image Translation Networks (2017, methodology): “VAEs were applied to translate face image attribute in (Yan et al., 2016).”
Semantic Image Inpainting with Deep Generative Models (2016, background): “, extensions to attribute-based image editing tasks [39].”
Coupled Generative Adversarial Networks (2016, methodology): “Note that our work is different to the Attribute2Image work[27], which is based on a conditional VAE model [28].”
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks (2016, methodology): “Most methods utilized simple conditioning variables such as attributes or class labels [37, 34, 4, 22].”
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks (2017, methodology): “Most methods utilize simple conditioning variables such as attributes or class labels [7], [31], [45], [52].”
Stacked Generative Adversarial Networks (2016, background): “More recently, several successful paradigms of deep generative models have emerged, including the auto-regressive models [32, 16, 58, 44, 45, 19], Variational Auto-encoders (VAEs) [28, 27, 50, 64, 18], and Generative Adversarial Networks (GANs) [17, 5, 47, 49, 53, 33].”
Photographic Image Synthesis with Cascaded Refinement Networks (2017, methodology): “[49] who use variational autoencoders and Mansimov et al.”
DISCO Nets : DISsimilarity COefficients Networks (2016, methodology): “presented in Goodfellow et al. [6] are very popular and have been used in several computer vision applications, for example in Denton et al. [1], Radford et al. [20], Springenberg [24] and Yan et al. [27]. A GAN model consists of two networks, simultaneously trained in an adversarial manner. A gener”
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation (2017, background): “Soon after, various conditional GANs (cGAN) have been proposed to condition the image generation on class labels [13], attributes [14, 24], texts [15], and images [7, 8, 21, 12, 4, 18].”
LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation (2016, methodology): “…the GANs framework is largely agnostic to the choice of G and D , it is clear that generative models with the ‘right’ inductive biases will be more effective in learning from the gradient information (Denton et al., 2015; Im et al., 2016; Gregor et al., 2015; Reed et al., 2016a; Yan et al., 2015).”
Fader Networks: Manipulating Images by Sliding Attributes (2017, background): “This task and the related problem of unsupervised domain transfer recently received a lot of interest [18, 25, 10, 27, 22, 24], as a case study for conditional generative models but also for applications like automatic image edition.”
Improved Variational Autoencoders for Text Modeling using Dilated Convolutions (2017, methodology): “Since effective variational techniques have been developed for learning VAEs (their namesake) (Kingma & Welling, 2013), these models have been successfully applied to image modeling and generation (Gregor et al., 2015; Salimans et al., 2015; Yan et al., 2016).”
Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts (2016, background): “Given a single input image, DC-IGN [12] generates new images of the same object with variations in pose and lighting and disCVAE [30] conditions the image generation process with facial attributes.”
Scribbler: Controlling Deep Image Synthesis with Sketch and Color (2016, background): “Examples of control signals include 3d pose of objects [8], natural language [39], semantic attributes [50], semantic segmentation [5], and object key-”
Pixel-Level Domain Transfer (2016, methodology): “However, training the converter is not straightforward because the target is not deterministic [15].”