Deep Attentive Video Summarization With Distribution Consistency Learning

By Zhong Ji, Yuxiao Zhao, Yanwei Pang, Xi Li, Jungong Han. 2020

83 citations recorded by the source corpus.

This article studies supervised video summarization by formulating it into a sequence-to-sequence learning framework, in which the input and output are sequences of original video frames and their predicted importance scores, respectively. Two critical issues are addressed in this article: short-term contextual attention insufficiency and distribution inconsistency. The former lies in the insufficiency of capturing the short-term contextual attention information within the video sequence itself since the existing approaches focus a lot on the long-term encoder–decoder attention. The latter refers to the distributions of predicted importance score sequence and the ground-truth sequence is inconsistent, which may lead to a suboptimal solution. To better mitigate the first issue, we incorporate a self-attention mechanism in the encoder to highlight the important keyframes in a short-term context. The proposed approach alongside the encoder–decoder attention constitutes our deep attentive models for video summarization. For the second one, we propose a distribution consistency learning method by employing a simple yet effective regularization loss term, which seeks a consistent distribution for the two sequences. Our final approach is dubbed as Attentive and Distribution consistent video Summarization (ADSum). Extensive experiments on benchmark data sets demonstrate the superiority of the proposed ADSum approach against state-of-the-art approaches.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Po-Yao (Bernie) Huang
2. Zhihui Li
3. Xiaojun Chang
4. Xin Wang
5. Xiaojiang Chen
6. Yun Xiao
7. Pengzhen Ren

In Signal Processing

1. Xiaojun Chang
2. Po-Yao (Bernie) Huang

Citations in the corpus

• A Comprehensive Survey of Neural Architecture Search (2020, background): “Artificially designed networks have made breakthroughs in many fields, including image recognition [4, 7, 112], machine translation [1, 3, 141, 149], semantic segmentation [142, 143, 148], object detection [8, 146, 147], and video understanding [144, 145] and so on.”