WordsEye: an automatic text-to-scene conversion system

By Robert Coyne, Richard Sproat. 2001

497 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Control and Systems Engineering, Multimodal Learning, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Multimodal Learning

In Natural Language Generation

Citations in the corpus

Scaling Autoregressive Models for Content-Rich Text-to-Image Generation (2022, methodology): “In this light, it is also worth recalling WordsEye [66], an automatic text-to-scene system built in 2001.”
Attribute2Image: Conditional Image Generation from Visual Attributes (2015, background): “Recent advances on representation learning using deep neural networks [16,29] nourish a series of deep generative models that enjoy joint generative modeling and representation learning through Bayesian inference [34,1,28,15,14,9] or adversarial training [8,3].”
Text-to-Image Generation Grounded by Fine-Grained User Attention (2020, background): “Such interfaces would support various applications, allowing users to rapidly create interactive scenes without specialized software [6], assist the creative process of an artist [13], or generate realistic images that are more closely grounded to intended descriptions [33].”
Deep correlation for matching images and text (2015, background): “Automatically describing (resp. synthesising) visual data using (resp. from) natural language [5] [6] [11] [15] [16] [23] [25] [26] [29] [34] [35] [36] [42] [52] is one of the ultimate goals of computer vision (CV), natural language processing (NLP) and machine learning (ML).”
Multimodal Machine Learning: A Survey and Taxonomy (2017, background): “Photo-realistic image generation has been less explored, and is still in early stages [137], [178], however, there have been a number of attempts at generating abstract scenes [261], computer graphics [47], and talking heads [5].”
Zero-Shot Learning with Generative Latent Prototype Model (2017, background): “This supposition is inspired from the cognitive process of human beings, who have remarkably ability of generating various representations, e.g. images, audios, texts, from high-level category labels [17]–[19].”
Learning the Visual Interpretation of Sentences (2013, methodology): “For instance, what does it mean for an object to be “next to” or “above” another object [6, 19].”
Every Picture Tells a Story: Generating Sentences from Images (2010, unknown): “Coyne and Sproat describe an auto-illustration system that gives naive users a method to produce rendered images from free text descriptions (Wordseye; [16]; http://www.wordseye.com ).”
Semantic Parsing for Text to 3D Scene Generation (2014, result): “This is similar to most previous work on text to scene generation (Winograd, 1972; Coyne and Sproat, 2001) and limits handling of natural language.”
Zero-Shot Text-Guided Object Generation with Dream Fields (2021, background): “but we do not target complex scene generation [6, 8, 11, 13] partly because CLIP poorly encodes spatial relations [29,53].”