Adopting Abstract Images for Semantic Scene Understanding

By C. Lawrence Zitnick, Ramakrishna Vedantam, Devi Parikh. 2016

82 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

ShapeWorld - A new test methodology for multimodal language understanding (2017, background): “Abstract Clipart scenes have been used for image captioning (Zitnick et al., 2016; Zitnick and Parikh, 2013) and to balance existing VQA datasets (Zhang et al., 2016).”
VQA: Visual Question Answering (2015, background): “To attract researchers interested in exploring the high-level reasoning required for VQA, but not the low-level vision tasks, we create a new abstract scenes dataset [1, 48, 49, 50] containing 50K scenes.”
MovieQA: Understanding Stories in Movies through Question-Answering (2015, background): “This is in large part due to efforts in large-scale data collection such as Microsoft’s COCO [22], Flickr30K [46] and Abstract Scenes [50] providing tens to hundreds of thousand images with natural language captions.”
Yin and Yang: Balancing and Answering Binary Visual Questions (2015, methodology): “Although our approach of visual verification is applicable to real images, we choose to use abstract images [2, 3, 43, 42, 44] as a test bed because abstract scene images allow us to focus on high-level semantic reasoning.”
A Dataset and Exploration of Models for Understanding Video Data through Fill-in-the-Blank Question-Answering (2016, methodology): “It consists of 254,721 images from the MSCOCO [20] dataset plus imagery of cartoon-like drawings from an abstract scene dataset [46].”
Reasoning about Pragmatics with Neural Listeners and Speakers (2016, methodology): “This dataset was previously used for a variety of language and vision tasks (e.g. Ortiz et al. (2015), Zitnick et al. (2014)).”
Learning Common Sense through Visual Abstraction (2015, background): “Zitnick et al . also link the semantics of a scene to memorability and saliency of objects [36].”
Visual question answering: A survey of methods and datasets (2016, methodology): “This type of images has been used before for capturing common sense [20, 46, 75], learning models of interactions between people [4], generating scenes from natural language descriptions [103], and learning the semantic relevance of visual features [102, 104].”
CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning (2019, background): “Similar attempts have been made for scene understanding through abstract scenes (Zitnick et al., 2016), with more recently”
Measuring Machine Intelligence Through Visual Question Answering (2016, methodology): “To enable multiple avenues for researching VQA, we introduce abstract scenes into the dataset [2, 34, 35, 36].”
VisKE: Visual knowledge extraction and question answering by visual verification of relation phrases (2015, background): “Also, in [41] the problem of learning common sense knowledge from clip art images was studied.”
Cross-Lingual Transfer Learning for Question Answering (2019, background): “QA tasks on images (Zitnick et al., 2016), audio (Lee et al., 2018; Li et al., 2018) or video descriptions (Rohrbach et al., 2015) have been studied, but most focus on understanding text documents (Lai et al., 2017; Rajpurkar et al., 2016; Clark et al., 2018; Joshi et al., 2017; He et al., 2017;…”