Video Summarization with Long Short-Term Memory

By Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman. 2016

783 citations recorded by the source corpus.

We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots. Casting the problem as a structured prediction problem on sequential data, our main idea is to use Long Short-Term Memory (LSTM), a special type of recurrent neural networks to model the variable-range dependencies entailed in the task of video summarization. Our learning models attain the state-of-the-art results on two benchmark video datasets. Detailed analysis justifies the design of the models. In particular, we show that it is crucial to take into consideration the sequential structures in videos and model them. Besides advances in modeling techniques, we introduce techniques to address the need of a large number of annotated data for training complex learning models. There, our main idea is to exploit the existence of auxiliary annotated video datasets, albeit heterogeneous in visual styles and contents. Specifically, we show domain adaptation techniques can improve summarization by reducing the discrepancies in statistical properties across those datasets.

Subfields: Computer Vision and Pattern Recognition, Model Architectures, Summarization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Cordelia Schmid
  5. Yi Yang
  6. Ming-Hsuan Yang
  7. Xiaogang Wang
  8. Andrew Zisserman

In Model Architectures

  1. Marcus Rohrbach
  2. Trevor Darrell
  3. Anna Rohrbach
  4. Michael Lewis
  5. Lorenzo Torresani
  6. Luowei Zhou
  7. Sinisa Todorovic
  8. Xi Li

In Summarization

  1. Zhong Ji
  2. Yanwei Pang
  3. Xu Sun
  4. M. Zhou
  5. Ah-Hwee Tan
  6. Fei Sha
  7. Medhini Narasimhan
  8. Cheston Tan

Citations in the corpus