How Many Words Is a Picture Worth? Automatic Caption Generation for News Images

By Yansong Feng, Mirella Lapata. 2010

96 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Multimodal Learning

Citations in the corpus

Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics (2013, background): “Some work, notably in the natural language processing community, has focused on images in news articles (Feng & Lapata, 2008, 2010).”
Parsing Algebraic Word Problems into Equations (2015, background): “More closely related is work on language grounding, whose goal is the interpretation of a sentence in the context of a world representation (Branavan et al., 2009; Liang et al., 2009; Chen et al., 2010; Bordes et al., 2010; Feng and Lapata, 2010; Hajishirzi et al., 2011; Matuszek et al., 2012; Hajis”
ReferItGame: Referring to Objects in Photographs of Natural Scenes (2014, methodology): “Retrieval based methods (Aker and Gaizauskas, 2010; Farhadi et al., 2010; Ordonez et al., 2011; Feng and Lapata, 2010; Feng and Lapata, 2013) retrieve existing captions or phrases to describe a query image.”
Choosing Linguistics over Vision to Describe Images (2012, neutral): “In Feng and Lapata (2010), assuming that a relevant document is available for a given image, the output (keywords) of an image annotation model is combined with the document properties to generate captions for images in the news domain.”
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation (2017, methodology): “A different view of retrieval is proposed by Feng and Lapata (2010), who use extractive summarisation techniques to retrieve descriptions of images and associated narrative fragments from their surrounding text in news articles.”
See No Evil, Say No Evil: Description Generation from Densely Labeled Images (2014, methodology): “…2 Available at : http://homes.cs.washington.edu/˜my89/ et al., 2010; Ordonez et al., 2011), using text surrounding an image in a news article (Feng and Lapata, 2010), ﬁlling visual sentence templates (Kulkarni et al., 2011; Yang et al., 2011; Elliott and Keller, 2013), and stitching…”
Translating Video Content to Natural Language Descriptions (2013, background): “(1) Assuming the availability of text associated with the image at test time one can effectively use summarization techniques [1, 7] which benefit from visual content.”
Collective Generation of Natural Image Descriptions (2012, unknown): “We employ Integer Linear Programming (ILP) as an optimization framework that has been used successfully in other generation tasks (e.g., Clarke and Lapata (2006), Mar-tins and Smith (2009), Woodsend and Lapata (2010)).”
Learning the Visual Interpretation of Sentences (2013, background): “That is, an image may be given as input and a sentence produced [8, 1, 13, 25, 38, 19], or a scene or animation may be generated from a sentence description [27, 6, 17, 14].”
Generalizing Image Captions for Image-Text Parallel Corpus (2013, background): “Other work (e.g. Feng and Lapata (2010a), Feng and Lapata (2010b)) uses computer vision to bias summarization of text associated with images to produce descriptions.”
Visual to Text: Survey of Image and Video Captioning (2019, methodology): “1) Image Datasets: The major datasets for evaluating the image caption generation performance include the BBC News dataset [71], the UIUC Pascal Dataset [43], the SBU Captioned Photo dataset [44], the Flickr30k Images dataset [115], and the Microsoft COCO (MS-COCO) dataset [116].”
Deep Learning in Medical Imaging: General Overview (2017, background): “Inspired by early research on image caption generation (78-80), studies have recently introduced the use of CNNs and RNNs (60-68) to combine recent advances in computer vision and machine translation, and thus automatically annotate chest radiographs with diseases and descriptions of the context of ”
Domain-Specific Image Captioning (2014, methodology): “The MixLDA model (Feng and Lapata, 2010b; Feng and Lapata, 2010a) learns from news images and natural language descriptions, but to generate words for a new image it requires both a query image and query text in the form of a news article.”
BabyTalk: Understanding and Generating Simple Image Descriptions (2013, methodology): “The process of generation then becomes one of combining or summarizing relevant documents, in some cases driven by keywords estimated from the image content [18].”
Baby Talk: Understanding and Generating Image Descriptions (2011, methodology): “For instance, [1] relies on GPS meta data to access relevant text documents and [13]”
Natural Language Models for Predicting Programming Comments (2013, background): “A related and well studied NLP task is that of predicting natural language caption and commentary for images and videos (Blei and Jordan, 2003; Feng and Lapata, 2010; Feng and Lapata, 2013; Wu and Li, 2011).”