Bringing Semantics into Focus Using Visual Abstraction

By Larry Zitnick, Devi Parikh. 2013

210 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

ShapeWorld - A new test methodology for multimodal language understanding (2017, background): “Abstract Clipart scenes have been used for image captioning (Zitnick et al., 2016; Zitnick and Parikh, 2013) and to balance existing VQA datasets (Zhang et al., 2016).”
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning (2016, background): “VQA [4] contains 150,000 natural-language questions about abstract scenes [50], but these questions do not control for questionconditional bias and are not equipped with functional program representations.”
VQA: Visual Question Answering (2015, methodology): “To attract researchers interested in exploring the high-level reasoning required for VQA, but not the low-level vision tasks, we create a new abstract scenes dataset [1, 48, 49, 50] containing 50K scenes.”
CIDEr: Consensus-based image description evaluation (2014, methodology): “The ABSTRACT-50S dataset is based on the dataset of Zitnick and Parikh [47].”
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models (2015, background): “Zitnick and Parikh (2013) have introduced the Abstract Scene dataset, which contains 10,020 synthetic images created using clip art objects from 58 categories, together with captions and ground-truth information of how objects relate to the captions.”
A Corpus for Reasoning about Natural Language Grounded in Photographs (2018, background): “However, commonly used resources (e.g., Zitnick and Parikh, 2013; Antol et al., 2015; Chen et al., 2016) are far from reﬂect-ing the full complexity of the problem.”
From captions to visual concepts and back (2014, background): “Determining the salient content requires not only knowing the contents of an image, but also deducing which aspects of the scene may be interesting or novel through commonsense knowledge [48, 4, 6].”
Yin and Yang: Balancing and Answering Binary Visual Questions (2015, methodology): “Some of these works have also taken advantage of visual abstraction to “control” the distribution of data, for example, [3] collects equal number of examples for each verb/preposition combinations, and [42] have multiple scenes that depict the exact same sentence/caption.”
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding (2016, background): “In terms of the approach, Hollywood in Homes is also related to [27].”
Learning a Recurrent Visual Representation for Image Caption Generation (2014, background): “[38] uses abstract clip art images to learn the visual interpretation of sentences.”
Unsupervised Semantic Parsing of Video Collections (2015, methodology): “[63, 62] used abstracted clip-arts to understand spatial relations of objects and their language correspondences.”
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation (2017, background): “Ortiz et al. (2015) use ilp to identify pairs of objects in abstract scenes (Zitnick & Parikh, 2013) before mapping them to a vdr. Realisation is framed as a machine translation task over vdr-text pairs.”
Don't just listen, use your imagination: Leveraging visual common sense for non-visual tasks (2015, methodology): “) are relevant to the meaning of a scene [51] and to learn spatiotemporal common sense to model scene dynamics [18].”
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures (2016, methodology): “The Abstract Scenes dataset (Zitnick & Parikh, 2013; Zitnick, Parikh, & Vanderwende, 2013) consists of 10,000 clip-art images and their descriptions.”
Neural Scene De-rendering (2017, methodology): “We start by describing the experimental setup; we then show how our framework performs on scene de-rendering with two different renderers, one used in the Abstract Scene dataset [39], and the other used in Minecraft.”
TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments (2018, background): “The most related resources to ours are R2R [2] and Talk the Walk [10].”
Reasoning about Pragmatics with Neural Listeners and Speakers (2016, methodology): “In particular, we construct instances of RG using the Abstract Scenes Dataset introduced by Zitnick and Parikh (2013).”