Long-Term Recurrent Convolutional Networks for Visual Recognition and Description

By Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, Trevor Darrell. 2017

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Long-term recurrent convolutional networks for visual recognition and description (2014, background): “The simplest strategy, and the one employed for most of our generation results in our prior work [43], is to generate captions greedily; i.”
CIDEr: Consensus-based image description evaluation (2014, background): “METEOR is known to have a correlation of ∼ 52 [9].”
Translating Videos to Natural Language Using Deep Recurrent Neural Networks (2014, methodology): “However, LSTMs (Hochreiter and Schmidhuber, 1997),”
Generation and Comprehension of Unambiguous Object Descriptions (2015, methodology): “From a modeling perspective, our approach is closest to recent works applying RNNs and CNNs to this problem domain [48, 9, 5, 12, 24, 26, 37, 50].”
Learning to Answer Questions from Image Using Convolutional Neural Network (2015, background): “…2014; Klein et al., 2015; Karpathy et al., 2014a; Ordonez et al., 2011) to automatic image captioning (Chen et al., 2014; Karpathy et al., 2014b; Donahue et al., 2014; Fang et al., 2014; Kiros et al., 2014a; Kiros et al., 2014b; Klein et al., 2015; Mao et al., 2014a; Mao et al., 2014b; Vinyals…”
Learning to predict the cosmological structure formation (2018, background): “Deep learning (11) is a fast growing branch of machine learning where recent advances have lead to models that reach and sometimes exceed human performance across diverse areas, from analysis and synthesis of images (12–14), sound (15, 16), text (17, 18) and videos (19, 20) to complex control and Fi”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, methodology): “We can use CNN to extract frame-based features and then conduct temporal fusion [45], or adopt 3D filter and 3D pooling layers to learn motion features [47], or use optical/scene flow to extract motion information [56, 22], or encode the video into images [51, 115, 21], or use RNN/LSTM to model the ”
Using Descriptive Video Services to Create a Large Data Source for Video Annotation Research (2015, background): “DVS narrations audio track on DVD is a rich source of data that describes videos in professionally written natu-ral sentences.”
Natural Language Generation in Dialogue using Lexicalized and Delexicalized Data (2016, background): “Bengio, Ducharme, and Vincent (2000) were some of the ﬁrst to propose using RNNs for language modelling.”
Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation (2017, methodology): “…as the COCO dataset (Lin et al., 2014), Visual features combined with language modeling have shown good performance both in image captioning (Devlin et al., 2015; Fang et al., 2014; Donahue et al., 2014) and in Visual Question Answering (VQA) (Antol et al., 2015) and (Malinowski and Fritz, 2014).”
Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model (2016, background): “Long Short-Term Memory networks [56] have achieved good performances on several tasks in which time dependencies are a key component [57], [58], [59], [60], but they can not be directly employed for saliency prediction, as they work on sequences of time varying vectors.”