Video Summarization Using Deep Neural Networks: A Survey

By Evlampios Apostolidis, Eleni Adamantidou, Alexandros I. Metsai, Vasileios Mezaris, Ioannis Patras. 2021

305 citations recorded by the source corpus.

Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content. Several approaches have been developed over the last couple of decades, and the current state of the art is represented by methods that rely on modern deep neural network architectures. This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization. After presenting the motivation behind the development of technologies for video summarization, we formulate the video summarization task and discuss the main characteristics of a typical deep-learning-based analysis pipeline. Then, we suggest a taxonomy of the existing algorithms and provide a systematic review of the relevant literature that shows the evolution of the deep-learning-based video summarization technologies and leads to suggestions for future developments. We then report on protocols for the objective evaluation of video summarization algorithms, and we compare the performance of several deep-learning-based approaches. Based on the outcomes of these comparisons, as well as some documented considerations about the amount of annotated data and the suitability of evaluation protocols, we indicate potential future research directions.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, methodology): “Our system draws similarity to early work in the area involving text-based summarization and identifying key frames (see (Barbieri, Agnihotri, and Dimitrova 2003) for an early survey and (Del Molino et al. 2016; Apostolidis et al. 2021) for more recent surveys).”
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding (2023, background): “There have been miscellaneous time-sensitive video tasks, including temporal video grounding [27, 49], dense video captioning [46, 52, 62], video summarization [1, 8, 41, 58], video highlight detection [15, 30], and step localization [5, 39, 55], etc.”