Title Generation for User Generated Videos

By Kuo-Hao Zeng, Tseng-Hung Chen, Juan Carlos Niebles, Min Sun. 2016

73 citations recorded by the source corpus.

A great video title describes the most salient event compactly and captures the viewer's attention. In contrast, video captioning tends to generate sentences that describe the video as a whole. Although generating a video title automatically is a very useful task, it is much less addressed than video captioning. We address video title generation for the first time by proposing two methods that extend state-of-the-art video captioners to this new task. First, we make video captioners highlight sensitive by priming them with a highlight detector. Our framework allows for jointly training a model for title generation and video highlight localization. Second, we induce high sentence diversity in video captioners, so that the generated titles are also diverse and catchy. This means that a large number of sentences might be required to learn the sentence structure of titles. Hence, we propose a novel sentence augmentation method to train a captioner with additional sentence-only examples that come without corresponding videos. We collected a large-scale Video Titles in the Wild (VTW) dataset of 18100 automatically crawled user-generated videos and titles. On VTW, our methods consistently improve title prediction accuracy, and achieve the best performance in both automatic and human evaluation. Finally, our sentence augmentation method also outperforms the baselines on the M-VAD dataset.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Multimodal Learning, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Multimodal Learning

In Natural Language Generation

Citations in the corpus

Video Summarization Using Deep Neural Networks: A Survey (2021, methodology): “Some less-commonly used datasets for video summarization are CoSum [82], MED-summaries [83], Video Titles in the Wild (VTW) [84], League of Legends (LoL) [85] and FVPSum [58].”
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding (2023, background): “, MovieQA [79], MVAD [81], MPII-MD [66], and MovieNet [32]) from different sources such as YouTube [14,58,105], Twitter [5–8], Flirck [5, 6], and internet [9].”
Less Is More: Picking Informative Frames for Video Captioning (2018, methodology): “Different from the previous work [46] that summarizing the video before video captioning, our method selects frames under partially observed settings and do not need any auxiliary annotation or information.”
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, background): “VTW26 (Zeng et al., 2016) is a large-scale dataset of automatically crawled user-generated YouTube videos paired with titles and descriptions.”
Multimodal Abstractive Summarization for How2 Videos (2019, background): “…which focus on events or activities in the video and generating descriptions at various levels of granularity from single sentence to multiple sentences (Das et al., 2013; Regneri et al., 2013; Rohrbach et al., 2014; Zeng et al., 2016; Zhou et al., 2018; Zhang et al., 2018; Gella et al., 2018).”
Leveraging Video Descriptions to Learn Video Question Answering (2016, methodology): “5 and more examples in technical report (Zeng et al. 2016a).”
In Search of a Lightweight "Good Enough" Offline Generative AI for Mobile Robots: Performance Benchmarking (2025, unknown): “To compare light-weight models, we performed two common NLP tasks: creating titles [10-12] and keywords [13-14] based on the abstracts of scholarly articles.”