DRAW: A Recurrent Neural Network For Image Generation

By Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, Daan Wierstra. 2015

2,013 citations recorded by the source corpus.

This paper introduces the Deep Recurrent Attentive Writer (DRAW) neural network architecture for image generation. DRAW networks combine a novel spatial attention mechanism that mimics the foveation of the human eye, with a sequential variational auto-encoding framework that allows for the iterative construction of complex images. The system substantially improves on the state of the art for generative models on MNIST, and, when trained on the Street View House Numbers dataset, it generates images that cannot be distinguished from real data with the naked eye.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Can Active Memory Replace Attention? (2016, background): “Finally, we discuss when active memory brings most benefits and where attention can be a better choice.”
Layer Normalization (2016, background): “Deep Recurrent Attention Writer (DRAW) [Gregor et al., 2015] has previously achieved the state-of-the-art performance on modeling the distribution of MNIST digits.”
Effective Approaches to Attention-based Neural Machine Translation (2015, background): “It is important to note that pt is a real nummber; whereas s is an integer within the window centered at pt.10 Comparison to (Gregor et al., 2015) – have proposed a selective attention mechanism, very similar to our local attention, for the image generation task.”
Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions (2017, background): “(2016) develop a sequential generative model for few-shot learning, generalizing the Deep Recurrent Attention Writer (DRAW) model (Gregor et al., 2015).”
Adaptive Computation Time for Recurrent Neural Networks (2016, background): “ACT promises to be particularly interesting for recurrent architectures containing soft attention modules [2, 10, 34, 12], which it could enable to dynamically adapt the number of glances or internal operations they perform at each time-step.”
Generating Sentences from a Continuous Space (2015, methodology): “Drawing inspiration from recent successes in modeling images (Gregor et al., 2015), handwriting, and natural speech (Chung et al., 2015), our model circum-vents these diﬃculties using the architecture of a variational autoencoder and takes advantage of recent advances in variational inference…”
Teaching Machines to Read and Comprehend (2015, background): “These models draw on recent developments for incorporating attention mechanisms into recurrent neural network architectures [6, 7, 8].”
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model (2017, background): “Finally, several previous works have tried to introduce latent variables into sequence modeling (Bayer & Osendorfer, 2014; Gregor et al., 2015; Chung et al., 2015; Gan et al., 2015; Fraccaro et al., 2016; Chung et al., 2016).”
Neural Autoregressive Distribution Estimation (2016, methodology): “Gregor et al. (2015) later exploited the VAE approach to develop DRAW, a directed generative model for images based on a read-write attentional mechanism.”
Pixel Recurrent Neural Networks (2016, neutral): “In (Gregor et al., 2015).”
Efficient Content-Based Sparse Attention with Routing Transformers (2020, background): “the selected area (Gregor et al., 2015; Xu et al., 2015; Luong et al., 2015). Later, hierarchical attention has been simplified by Liu et al. (2018) that alternates coarse layers (attending to the whole sequence at a lower temporal resolution) with local layers (attending to a neighborhood of the cu”
Object-Centric Learning with Slot Attention (2020, background): “[70] Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra.”
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks (2016, methodology): “Next, we consider DRAW, a recurrent generative model by [7].”
Spatial Transformer Networks (2015, methodology): “need for a differentiable attention mechanism, while [11] use a differentiable attention mechansim by utilising Gaussian kernels in a generative model.”
Zero-Shot Text-to-Image Generation (2021, background): “Modern machine learning approaches to text to image synthesis started with the work of Mansimov et al. (2015), who showed that the DRAW Gregor et al. (2015) generative model, when extended to condition on image captions, could also generate novel visual scenes.”
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors (2016, neutral): “This is similar to the differentiable cropping mechanism of [5], the attention model of [12] as well as the Spatial”
Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, methodology): “In terms of synthesis, various approaches based on the ConvNet have been proposed for synthesizing realistic static images [13], [14], [15], [16], [17].”