Generating images with recurrent adversarial networks

By Daniel Jiwoong Im, Chris Dongjoo Kim, Hui Jiang, Roland Memisevic. 2016

227 citations recorded by the source corpus.

Gatys et al. (2015) showed that optimizing pixels to match features in a convolutional network with respect reference image features is a way to render images of high visual quality. We show that unrolling this gradient-based optimization yields a recurrent computation that creates images by incrementally adding onto a visual "canvas". We propose a recurrent generative model inspired by this view, and show that it can be trained using adversarial training to generate very good image samples. We also propose a way to quantitatively compare adversarial networks by having the generators and discriminators of these networks compete against each other.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Improved Techniques for Training GANs (2016, background): “Several recent papers focus on improving the stability of training and the resulting perceptual quality of GAN samples [2, 3, 5, 6].”
Density estimation using Real NVP (2016, neutral): “Indeed, if g is bijective, it can be trained through maximum likelihood using the change of variable formula: pX(x) = pZ(z) ∣∣∣∣det(∂g(z)∂zT )∣∣∣∣−1 .”
Gradient descent GAN optimization is locally stable (2017, methodology): “…deployed for tasks such as photo super-resolution, object generation, video prediction, language modeling, vocal synthesis, and semi-supervised learning, amongst many others [Ledig et al., 2016, Wu et al., 2016, Mathieu et al., 2015, Nguyen et al., 2016, Denton et al., 2015, Im et al., 2016].”
Learning from Simulated and Unsupervised Images through Adversarial Training (2016, background): “[13] propose a recurrent generative model trained using adversarial training.”
On the Quantitative Analysis of Decoder-Based Generative Models (2016, background): “While alternative quantitative criteria have been proposed [4, 13, 25], log-likelihood of held-out test data remains one of the most important measures of a generative model’s performance.”
Energy-based Generative Adversarial Network (2016, background): “, 2014) have led to significant improvements in image generation (Denton et al., 2015; Radford et al., 2015; Im et al., 2016; Salimans et al., 2016), video prediction (Mathieu et al.”
Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks (2016, background): “These models can also be augmented semantic attributes [35], image captions [24], 3D data [5,17], spatial/temporal status [11,13,27] etc.”
Generative Image Modeling Using Style and Structure Adversarial Networks (2016, neutral): “There are more extensions include using conditional variables [36,37,38].”
Recent advances in convolutional neural networks (2015, methodology): “During forward propagation process, λ is recorded and will be used for the backpropagation operation.”
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling (2016, background): “To name a few, Wang and Gupta [2016] discussed how to model image style and structure with sequential GANs, Li and Wand [2016] and Zhu et al. [2016] used GAN for texture synthesis and image editing, respectively, and Im et al. [2016] z Figure 1: The generator in 3D-GAN.”
MoCoGAN: Decomposing Motion and Content for Video Generation (2017, methodology): “Recurrent neural networks for image generation were previously explored in [14, 16, 27].”
Age Progression/Regression by Conditional Adversarial Autoencoder (2017, background): “During the last two years, several approaches [21, 20, 9, 3, 4, 10, 19] have been proposed to improve the original GAN from different perspectives.”
Temporal Generative Adversarial Nets with Singular Value Clipping (2016, methodology): “Conditional GANs performs modeling of object attributes [22, 12].”
High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis (2016, background): “Meanwhile, researchers use deep neural networks for structure prediction [29, 4, 30, 7, 38, 17, 18, 21, 9, 31], semantic segmentation [29, 4, 30], and image generation [17, 18, 7, 31].”
Pros and Cons of GAN Evaluation Measures (2018, background): “[38] proposed to compare two GANs by having them engage in a battle against each other by swapping discriminators or generators across the two models (See Fig.”
A guide to convolution arithmetic for deep learning (2016, methodology): “Finally, so-called transposed convolutional layers (also known as fractionally strided convolutional layers) have been employed in more and more work as of late (Zeiler et al., 2011; Zeiler and Fergus, 2014; Long et al., 2015; Radford et al., 2015; Visin et al., 2015; Im et al., 2016), and their rel”