Mind's eye: A recurrent visual representation for image caption generation

By Xinlei Chen, C. Lawrence Zitnick. 2015

518 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Natural Language Generation

Citations in the corpus

VQA: Visual Question Answering (2015, methodology): “In particular, research in image and video captioning that combines Computer Vision (CV), Natural Language Processing (NLP), and Knowledge Representation & Reasoning (KR) has dramatically increased in the past year [13, 7, 9, 32, 21, 19, 45].”
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models (2015, methodology): “C V ] 1 5 A pr 2 dataset (Young et al. 2014), a popular benchmark for caption generation and retrieval that has been used, among others, by Chen and Zitnick (2015); Donahue et al. (2015); Fang et al. (2015); Gong et al. (2014b); Karpathy et al. (2014); Karpathy and Fei-Fei (2015); Kiros et al.…”
From captions to visual concepts and back (2014, background): “Among these, a common theme [29, 45, 6, 18] has been to utilize a recurrent neural network”
Movie Description (2016, background): “Recently image description has gained increased attention with work such as that of Chen and Zitnick (2015); Donahue et al. (2015); Fang et al. (2015); Karpathy and Fei-Fei (2015); Kiros et al. (2014, 2015); Mao et al. (2015); Vinyals et al. (2015); Xu et al. (2015a).”
Yin and Yang: Balancing and Answering Binary Visual Questions (2015, background): “In general, captions borrowed from nearest neighbor images result in a surprisingly high performance [8].”
DenseCap: Fully Convolutional Localization Networks for Dense Captioning (2015, methodology): “Following previous work [32, 21, 49, 8, 4], we use the region codes to condition an RNN language model [15, 33, 44].”
Grounding of Textual Phrases in Images by Reconstruction (2015, methodology): “[3] uses a bi-directional mapping from visual features to words and words to visual features in a recurrent neural network model to generate descriptions based on visual feature, and to reconstruct visual descriptions given a description.”
Exploring Nearest Neighbor Approaches for Image Captioning (2015, methodology): “Recently proposed methods for caption generation share many similarities, including the use of deep learned image features [20], [14], [32], and language models using maximum entropy [8], recurrent neural networks [2], [16], and LSTMs [35], [6].”
Learning Language-Visual Embedding for Movie Understanding with Natural-Language (2016, background): “In the last 1-2 years, several works have been developed for automatic image [3,4,5,10,11,30,26] and video [25,31] captioning.”
Image Captioning with Semantic Attention (2016, background): “The topdown paradigm [4, 35, 26, 16, 8, 37, 25] starts from a “gist” of an image and converts it into words, while the bottom-up one [12, 19, 23, 9, 20, 11, 22] first comes up with words describing various aspects of an image and then combines them.”
RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning (2020, background): “Recurrent latent variable is introduced in [5], encoding the visual interpretation of previously-generated words and acting as a long-term visual memory during next words generation.”
Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning (2016, methodology): “In general, those methods can be divided into two categories: template-based [9, 13, 14, 20] and neural-based [12, 18, 6, 3, 27, 7, 11, 30, 8, 34, 32, 33].”