A Sentence Is Worth a Thousand Pixels

By Sanja Fidler, Abhishek Sharma, Raquel Urtasun. 2013

56 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Jianfeng Gao
3. Li Fei-Fei
4. Trevor Darrell
5. Percy Liang
6. Ruslan Salakhutdinov
7. Quoc V. Le
8. Yoshua Bengio

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Trevor Darrell
3. Dacheng Tao
4. Jitendra Malik
5. Li Fei-Fei
6. Xiaogang Wang
7. Andrew Zisserman
8. Ming-Hsuan Yang

In Multimodal Learning

1. Marcus Rohrbach
2. Devi Parikh
3. Mohit Bansal
4. Svetlana Lazebnik
5. Sanja Fidler
6. Li Fei-Fei
7. Trevor Darrell
8. Liwei Wang

Citations in the corpus

• Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books (2015, methodology): “For text-to-image alignment, [17, 8] find correspondences between nouns and pronouns in a caption and visual objects using several visual and textual potentials.”
• Deep Visual-Semantic Alignments for Generating Image Descriptions (2014, neutral): “We first present a model that aligns segments of sentences to the visual regions that they describe through a multimodal embedding.”
• Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models (2015, background): “Because our dataset is densely annotated with multiple boxes per image linked to their textual mentions in a larger sentence context, it will also be a rich resource for learning models of multi-object spatial layout (Farhadi et al. 2010; Fidler et al. 2013; Kulkarni et al. 2011).”
• Unsupervised Semantic Parsing of Video Collections (2015, methodology): “[13] both accomplished the task of learning spatial reasoning using the image captions.”
• Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos (2017, background): “This includes recent progress in scene graphs [13, 20, 51, 64], storylines [1, 14, 17, 31, 53], and action understanding [10, 44, 55].”
• TVSum: Summarizing web videos using titles (2015, background): “This makes it particularly difficult to learn canonical visual concepts shared between video and images [19, 29].”
• Improving Image-Sentence Embeddings Using Large Weakly Annotated Photo Collections (2014, background): “, [1–6], use natural language models or templates for generating sentences, and learn predictors to “fill in” or compose parts of these models.”
• Multimodal Machine Learning: A Survey and Taxonomy (2017, methodology): “A CRF model was used to better segment images by combining visual and textual information of image description [63].”
• Linking People in Videos with "Their" Names Using Coreference Resolution (2014, methodology): “[13] used image descriptions to improve object detection and segmentation.”
• A High Performance CRF Model for Clothes Parsing (2014, methodology): “Most approaches combine detection and segmentation by using detectors as additional image evidence [21,22].”
• What Are You Talking About? Text-to-Image Coreference (2014, methodology): “An exception is [9], which employed nouns and prepositions extracted from short sentences to boost the performance of object detection and semantic segmentation.”
• Blending Learning and Inference in Conditional Random Fields (2016, methodology): “…Lin et al. (2013)), shape reconstruction (Salzmann and Urtasun (2012)) indoor scene understanding (Schwing et al. (2012a); Schwing and Urtasun (2012)), depth estimation (Yamaguchi et al. (2012)), flow estimation (Yamaguchi et al. (2013)) and visuallanguage understanding (Fidler et al. (2013)).”
• Visual Semantic Search: Retrieving Videos via Complex Textual Queries (2014, methodology): “Finally, we develop a matching algorithm, which takes into account both visual cues and spatial/semantic relations, to infer correspondence between the entities described in the query and the objects detected in the video.”
• Deep Visual-Semantic Alignments for Generating Image Descriptions (2017, background): “Several works [27] [12] [9] studied the problem of holistic scene understanding in which the scene type, objects and their spatial support in the image is inferred.”
• Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes (2016, unknown): “…the generality of our approach, and to show that our approach is compatible with a wide class of implementations of semantic segmentation and PPAR modules , we present our approach with the modules abstracted as “black boxes” that satisfy a few general requirements and minimal assumptions.”