A Thousand Frames in Just a Few Words: Lingual Description of Videos through Latent Topics and Sparse Object Stitching

By Pradipto Das, Chenliang Xu, Richard F. Doell, Jason J. Corso. 2013

331 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Towards Automatic Learning of Procedures From Web Instructional Videos (2017, methodology): “We demonstrate qualitative results with videos from YouCookII test set (see Fig.”
Long-term recurrent convolutional networks for visual recognition and description (2014, methodology): “[11, 30, 17, 3, 6, 17, 41, 42] propose methods for generating sentence descriptions for video, but to our knowledge we present the first application of deep models to the video description task.”
Dense-Captioning Events in Videos (2017, background): “. In an effort to map video semantics with language, MPII MD [39] and M-VAD [47] released short movie clips with descriptions. In an effort to capture longer events, MSRVTT [55], MSVD [6] and YouCook [7] collected a dataset with slightly longer length, at the cost of a few descriptions than previous”
MovieQA: Understanding Stories in Movies through Question-Answering (2015, background): “Most available datasets focus on image [17, 22, 46, 50] or video description [5, 32, 9].”
Movie Description (2016, background): “The main limitations of prior data-sets include the coverage of a single domain (Das et al. 2013; Regneri et al. 2013; Rohrbach et al. 2014) and having a limited number of video clips (Chen and Dolan 2011).”
End-to-End Dense Video Captioning with Masked Transformer (2018, neutral): “In contrast to earlier video captioning papers, which are based on models like hidden Markov models and ontologies [39, 6], recent work on captioning is dominated by deep neural network-based methods [32, 34, 37, 43, 36, 26].”
Translating Videos to Natural Language Using Deep Recurrent Neural Networks (2014, background): “Most of the existing research in video description has focused on narrow domains with limited vocabularies of objects and activities (Kojima et al., 2002; Lee et al., 2008; Khan and Gotoh, 2012; Barbu et al., 2012; Ding et al., 2012; Khan and Gotoh, 2012; Das et al., 2013b; Das et al., 2013a; Rohrba”
Localizing Moments in Video with Natural Language (2017, background): “g that descriptions in DiDeMo are referring expressions, meaning that they refer Dataset # Videos/# Clips # Sentences Video Source Domain Temporal Localization Un-Edited Referring Expressions YouCook [7] 88/- 2,668 YouTube Cooking Charades [38] 10,000/- 16,129 Homes Daily activities X TGIF [19] 100,”
Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks (2015, methodology): “To eliminate the tedious effort of rule engineering when the problem scales, some recent methods train statistical models for lexical entries, either in a fully [10, 14, 24, 41] or weakly [36, 35, 57, 54] supervised fashion.”
Jointly Modeling Deep Video and Compositional Text to Bridge Vision and Language in a Unified Framework (2015, background): “As (Das et al. 2013) discussed, the scalability of nonparametric methods with increasing of semantic space is unclear, and the generated text often lacks “semantic verification.”
Grounded Video Description (2018, methodology): “Early work on automatic caption generation mainly includes template-based approaches [5, 12, 18], where predefined templates with slots are first generated and then filled in with detected visual evidences.”
Zero-Shot Anticipation for Instructional Activities (2018, background): “In computer vision, cooking has been well-explored for complex and fine-grained activity recognition [30, 43, 17, 42, 16, 59], temporal segmentation [30, 59] and captioning [44, 39, 60].”