Enhancing Video Summarization via Vision-Language Embedding

By Bryan A. Plummer, Matthew A. Brown, Svetlana Lazebnik. 2017

113 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Self-Supervised MultiModal Versatile Networks (2020, background): “With larger datasets [48, 69, 71, 92, 96], many works have profited from learning such a joint visual-textual space [17, 19, 29, 30, 41, 53, 57, 63, 68, 84, 85, 87, 88, 93].”
End-to-End Learning of Visual Representations From Uncurated Instructional Videos (2019, background): “Numerous prior works [19, 21, 27, 28, 42, 51, 55, 60, 62, 88, 79, 80, 83, 84] have used image / video description datasets [48, 63, 65, 87, 90] to learn an embed-”
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips (2019, background): “These applications include automatic image and video captioning [20, 36, 60, 61], visual question answering [9, 29, 51, 63], visual content retrieval based on textual queries [32, 56, 63], temporal localization of events in videos using natural language [15, 26] or video summarization with natural l”
Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval (2018, background): “tic models have shown excellent performance on several multimedia tasks, e.g., cross-modal retrieval [18, 31, 38, 55], image captioning [23, 36], image classification [10, 12, 20] video summarization [4, 41]. Crossmodal retrieval methods require computing semantic similarity between twodifferent mod”
Learning a Text-Video Embedding from Incomplete and Heterogeneous Data (2018, background): “A useful application of learning jointly from video and text is the possibility of performing video summarization with natural language [8].”
Learning Joint Embedding with Multimodal Cues for Cross-Modal Video-Text Retrieval (2018, background): “Joint visual-semantic embeddings [9, 15, 20, 26, 33] underpin the building of most cross-modal retrieval methods as they can bridge the gap between different modalities.”
Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings (2019, background): “As is common in cross-modal search tasks [26, 36], we learn a shared embedding space onto which we project both videos and captions.”
A Weighted Sparse Sampling and Smoothing Frame Transition Approach for Semantic Fast-Forward First-Person Videos (2018, background): “By not including the very last and the following frames of a clip, the summarization might lose the clip context [20].”
Textually Customized Video Summaries (2017, methodology): “ry, but not over specic summary composition which is learned via reference summaries. Also, this approach is not applicable to novel nouns other than the predened set of nouns. A data driven approach [27] concurrent to our work takes a reference text summary as input to generate a video summary. It ”
CLIP-It! Language-Guided Video Summarization (2021, background): “[28] use image-language embeddings for generating video summaries but evaluate their approach in the text domain, and not on the generic video summarization benchmarks.”
Learning Spatiotemporal Features via Video and Text Pair Discrimination (2020, background): “h motion and audio information seem low-level signals and may lack highlevel semantic for cross-modal learning. Speech or text has been widely studied as another cross-modal setting in video learning [49,35,10,34,40,41]. These works mainly aimed to learn a joint video-text embedding where visual and”
Viewpoint-Aware Video Summarization (2018, background): “However, owing to the subjective property of this task, a supervised approach [21, 38, 32, 23, 12, 31, 13, 19, 9, 42], that trains a summarization model which takes human-created summaries as the supervision, became standard because of its better performance.”
Video Summarization Using Deep Neural Networks: A Survey (2021, background): “Some indicative and recent examples can be found in [61], [62], [63], [64], [65], [66], [67], [68].”
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning (2020, background): “Representation learning based on both vision and language has many potential benefits: visual grounding[1, 2, 3, 4]; visual learning with a more natural, almost self-supervised annotation process; and direct applicability to cross-modal tasks, such as video retrieval by text[5, 6, 7, 8, 9], video su”
Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers (2021, background): “A large number of vision and language retrieval models [15, 19, 20, 36, 50, 51, 55, 59, 71, 72, 74, 75, 77] use a dual encoder architecture where the text and vision inputs are separately embedded into a joint space.”