Textually Customized Video Summaries
By Jinsoo Choi, Tae-Hyun Oh, In-So Kweon. 2017
12 citations recorded by the source corpus.
The best summary of a long video differs among different people due to its highly subjective nature. Even for the same person, the best summary may change with time or mood. In this paper, we introduce the task of generating customized video summaries through simple text. First, we train a deep architecture to effectively learn semantic embeddings of video frames by leveraging the abundance of image-caption data via a progressive and residual manner. Given a user-specific text description, our algorithm is able to select semantically relevant video segments and produce a temporally aligned video summary. In order to evaluate our textually customized video summaries, we conduct experimental comparison with baseline methods that utilize ground-truth information. Despite the challenging baselines, our method still manages to show comparable or even exceeding performance. We also show that our method is able to generate semantically diverse video summaries by only utilizing the learned visual embeddings.
Subfields: Computer Vision and Pattern Recognition, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Rameswar Panda
- Niluthpol Chowdhury Mithun
- Evangelos E. Papalexakis
- Amit K. Roy-Chowdhury
- Florian Metze
- Juncheng Billy Li
- Cordelia Schmid
- Andrew Zisserman
In Natural Language Processing
- Amit K. Roy-Chowdhury
- Florian Metze
- Rameswar Panda
- Evangelos E. Papalexakis
- Juncheng Billy Li
- Niluthpol Chowdhury Mithun
Citations in the corpus
- Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval (2018, background): “Visual-Semantic Embedding: Joint visual-semantic models have shown excellent performance on several multimedia tasks, e.g., cross-modal retrieval [18, 31, 38, 55], image captioning [23, 36], image classification [10, 12, 20] video summarization [4, 41].”
- Learning Joint Embedding with Multimodal Cues for Cross-Modal Video-Text Retrieval (2018, background): “Joint visual-semantic models have shown impressive performance on several multimedia and computer vision tasks, such as cross-modal retrieval [21, 35], image captioning [16, 23], image classification [10] video summarization [5, 28].”