Learning to generate chairs with convolutional neural networks

By Alexey Dosovitskiy, Jost Tobias Springenberg, Thomas Brox. 2014

670 citations recorded by the source corpus.

We train a generative convolutional neural network which is able to generate images of objects given object type, viewpoint, and color. We train the network in a supervised manner on a dataset of rendered 3D chair models. Our experiments show that the network does not merely learn all images by heart, but rather finds a meaningful representation of a 3D chair model allowing it to assess the similarity of different chairs, interpolate between given viewpoints to generate the missing ones, or invent new chair styles by interpolating between chairs from the training set. We show that the network can be used to find correspondences between different chairs from the dataset, outperforming existing approaches on this task.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (2016, background): “Similarly, VAEs [5] and Adversarial Autoencoders [9] were shown to learn representations in which class label is separated from other variations.”
Context Encoders: Feature Learning by Inpainting (2016, background): “It can be understood as upsampling followed by convolution (as described in [10]), or convolution with fractional stride (as described in [28]).”
Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, methodology): “In terms of synthesis, various approaches based on the ConvNet have been proposed for synthesizing realistic static images [13], [14], [15], [16], [17].”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, background): “The properties of the latent space are also poorly understood, and the commonly demonstrated latent space interpolations [12, 48, 34] provide no quantitative way to compare different generators against each other.”
Generative Adversarial Text to Image Synthesis (2016, methodology): “Dosovitskiy et al. (2015) trained a deconvolutional network (several layers of convolution and upsampling) to generate 3D chair renderings conditioned on a set of graphics codes indicating shape, position and lighting.”
Generating Images with Perceptual Similarity Metrics based on Deep Networks (2016, methodology): “By combining convolutions and un-pooling (upsampling) layers [5, 1, 8] these models can be applied to large images.”
Generative Visual Manipulation on the Natural Image Manifold (2016, background): “, [19]) and their deep variants [20], auto-encoders [19,21] and more recently, stochastic neural networks [22,3,23] and deterministic networks [24].”
Coupled Generative Adversarial Networks (2016, background): “Our work is also related to the prior works in cross-domain image generation [31, 32, 33], which studied transforming an image in one style to the corresponding images in another style.”
Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks (2015, methodology): “They are similar to the models from Rasmus et al. (2015) who, in turn, derived them from the best models found by Springenberg et al. (2015).”
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (2015, background): “A recurrent network approach (Gregor et al., 2015) and a deconvolution network approach (Dosovitskiy et al., 2014) have also recently had some success with generating natural images.”
Deep multi-scale video prediction beyond mean square error (2015, methodology): “Another method is to use connections that “skip” the pooling/unpooling pairs, to preserve the high frequency information (Long et al., 2015; Dosovitskiy et al., 2015; Ronneberger et al., 2015).”
A note on the evaluation of generative models (2015, background): “It is therefore unsurprising that other approaches than density estimation are much more effective for image synthesis (Portilla & Simoncelli, 2000; Dosovitskiy et al., 2015; Gatys et al., 2015).”
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images (2015, methodology): “We consider two different network types for our model: Standard fully connected neural networks with up to three layers which work well for moderately sized images and are used for the planar and swing-up experiments; A deep convolutional network for the encoder in combination with an “up-convolutio”
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks (2016, background): “[5] trained deconvolutional neural networks to generate 3D chairs, tables and cars.”
Action-Conditional Video Prediction using Deep Networks in Atari Games (2015, background): “3 Convolutional Decoding It has been recently shown that a CNN is capable of generating an image given fully-specified attributes of the image [1].”
Autoencoding beyond pixels using a learned similarity metric (2015, methodology): “First, Dosovitskiy et al. (2015) used supervised training to train convolutional network to generate chairs given high-level information about the desired chair.”