Generating Image Descriptions Using Dependency Relational Patterns

By Ahmet Aker, Robert J. Gaizauskas. 2010

103 citations recorded by the source corpus.

Subfields: Information Extraction, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Extraction

In Natural Language Generation

Citations in the corpus

Show and tell: A neural image caption generator (2014, background): “…memory at time t via the three gates; the cell value is fed back via the forget gate; the predicted word at time t− 1 is fed back in addition to the memory output m at time t into the Softmax for word prediction. read its input (input gate i) and whether to output the new cell value (output gate…”
ReferItGame: Referring to Objects in Photographs of Natural Scenes (2014, methodology): “Retrieval based methods (Aker and Gaizauskas, 2010; Farhadi et al., 2010; Ordonez et al., 2011; Feng and Lap-ata, 2010; Feng and Lapata, 2013) retrieve existing captions or phrases to describe a query image.”
Visual Madlibs: Fill in the blank Image Generation and Question Answering (2015, methodology): “The second type of approach explored methods to make use of existing text either directly associated with an image [11, 1] or retrieved from visually similar images [27, 19, 24].”
Translating Videos to Natural Language Using Deep Recurrent Neural Networks (2014, background): “The recurrences for the LSTM are then deﬁned as: where σ is the sigmoidal non-linearity, φ is the hyperbolic tangent non-linearity, represents the product with the gate value, and the weight matrices denoted by W ij are the trained parameters.”
Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge (2016, methodology): “An “encoder” RNN reads the source sentence and transforms it into a rich ﬁxed-length vector representation, which in turn in used as the initial hidden state of a “decoder” RNN that generates the target sentence.”
Choosing Linguistics over Vision to Describe Images (2012, neutral): “Extracting dependency patterns from descriptions rather than us-Proceedings ing it as an n-gram language model is inspired by Aker and Gaizauskas (2010).”
What Value Do Explicit High Level Concepts Have in Vision to Language Problems? (2015, methodology): “Instead of using ﬁxed templates, more powerful language models based on language parsing have been developed, such as [1, 27, 28, 39].”
A Comprehensive Survey of Deep Learning for Image Captioning (2018, methodology): “mmatically correct captions. However, templates are predefined and cannot generate variable-length captions. Moreover, later on, parsing based language models have been introduced in image captioning [2, 32, 76, 77, 101] which are more powerful than fixed template-based methods. Therefore, in this p”
Composing Simple Image Descriptions using Web-scale N-grams (2011, methodology): “Our work contrasts to most previous approaches in four key aspects: ﬁrst, we compose fresh sentences from scratch, instead of retrieving (Farhadi et al. (2010)), or summarizing existing text fragments associated with an image (e.g., Aker and Gaizauskas (2010), Feng and Lapata (2010a)).”
Visual Madlibs: Fill in the Blank Description Generation and Question Answering (2015, background): “Early work in this area focused on detecting content elements and then composing captions [20, 36, 28, 11, 18] or made use of existing text either directly associated with an image [12, 1] or retrieved from visually similar images [29, 21, 26].”
BabyTalk: Understanding and Generating Simple Image Descriptions (2013, methodology): “For instance, Aker and Gaizauskas [1] rely on GPS metadata to access relevant text documents and Feng and Lapata [18] assume relevant documents are provided.”
Nonparametric Method for Data-driven Image Captioning (2014, background): “Another line of related work (Fan et al., 2010; Aker and Gaizauskas, 2010; Feng and Lapata, 2010) generates captions by extracting sentences from documents which are related to the query image.”
Translating Video Content to Natural Language Descriptions (2013, background): “(1) Assuming the availability of text associated with the image at test time one can effectively use summarization techniques [1, 7] which benefit from visual content.”
Learning the Visual Interpretation of Sentences (2013, background): “That is, an image may be given as input and a sentence produced [8, 1, 13, 25, 38, 19], or a scene or animation may be generated from a sentence description [27, 6, 17, 14].”
Automatic Caption Generation for News Images (2013, background): “More recently, Aker and Gaizauskas [31] generated extended captions for images indexed by GPS coordinates (e.g., a multisentence description of the Eiffel Tower, including where and when it was built and by whom, why it is an important site, and so on).”
Collective Generation of Natural Image Descriptions (2012, unknown): “Some past research has simpliﬁed the general image description goal by assuming that relevant text for an image is provided (e.g., Aker and Gaizauskas (2010), Feng and Lapata (2010)).”
From Show to Tell: A Survey on Deep Learning-Based Image Captioning (2021, background): “entailed description retrieval [2], [3], [4], [5], [6], [7] or template filling and hand-crafted natural language generation techniques [8], [9], [10], [11], [12], [13], [14], [15].”
Image Captioning and Visual Question Answering Based on Attributes and Their Related External Knowledge (2016, background): “Instead of us-ing ﬁxed templates, more powerful language models based on language parsing have been developed, such as [38], [39], [40], [41].”
Baby Talk: Understanding and Generating Image Descriptions (2011, methodology): “For instance, [1] relies on GPS meta data to access relevant text documents and [13]”