Predicting Important Objects for Egocentric Video Summarization

By Yong Jae Lee, Kristen Grauman. 2015

166 citations recorded by the source corpus.

We present a video summarization approach for egocentric or “wearable” camera data. Given hours of video, the proposed method produces a compact storyboard summary of the camera wearer’s day. In contrast to traditional keyframe selection techniques, the resulting summary focuses on the most important objects and people with which the camera wearer interacts. To accomplish this, we develop region cues indicative of high-level saliency in egocentric video—such as the nearness to hands, gaze, and frequency of occurrence—and learn a regressor to predict the relative importance of any new region based on these cues. Using these predictions and a simple form of temporal event detection, our method selects frames for the storyboard that reflect the key object-driven happenings. We adjust the compactness of the final summary given either an importance selection criterion or a length budget; for the latter, we design an efficient dynamic programming solution that accounts for importance, visual uniqueness, and temporal displacement. Critically, the approach is neither camera-wearer-specific nor object-specific; that means the learned importance metric need not be trained for a given user or context, and it can predict the importance of objects and people that have never been seen previously. Our results on two egocentric video datasets show the method’s promise relative to existing techniques for saliency and summarization.

Subfields: Computer Vision and Pattern Recognition, Summarization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Kristen Grauman
  2. Luc Van Gool
  3. Jitendra Malik
  4. Trevor Darrell
  5. Dacheng Tao
  6. Andrew Zisserman
  7. Cordelia Schmid
  8. Ming-Hsuan Yang

In Summarization

  1. Ah-Hwee Tan
  2. Cheston Tan
  3. Ana Garcia Del Molino
  4. Joo-Hwee Lim
  5. Tushar Nagarajan
  6. James M. Rehg
  7. Miao Liu
  8. Yin Li

Citations in the corpus