Generating Sentences Using a Dynamic Canvas

By Harshil Shah, Bowen Zheng, David Barber. 2018

8 citations recorded by the source corpus.

We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word level generative model for natural language. It uses a recurrent neural network with a dynamic attention and canvas memory mechanism to iteratively construct sentences. By viewing the state of the memory at intermediate stages and where the model is placing its attention, we gain insight into how it constructs sentences. We demonstrate that AUTR learns a meaningful latent representation for each sentence, and achieves competitive log-likelihood lower bounds whilst being computationally efficient. It is effective at generating and reconstructing sentences, as well as imputing missing words.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Generative Models

Citations in the corpus

BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling (2019, background): “Nonetheless, autoregressive components remain a cornerstone of the generative models for text [2, 48, 49].”
Insertion Transformer: Flexible Sequence Generation via Insertion Operations (2019, background): “Shah et (2018) also recently proposed generating language with a dynamic canvas.”
Learning to Generate Code Sketches (2021, background): “Recently, sequence generation approaches that go beyond the left-to-right paradigm have been proposed [37, 32, 11, 10, 17, 30], usually by considering generation as an iterative reﬁnement procedure that changes or extends a sequence in every iteration.”