Information theory-based shot cut/fade detection and video summarization

By Z. Černeková, Ioannis Pitas, Christophoros Nikou. IEEE transactions on circuits and systems for video technology (Print), 2006

333 citations recorded by the source corpus.

New methods for detecting shot boundaries in video sequences and for extracting key frames using metrics based on information theory are proposed. The method for shot boundary detection relies on the mutual information (MI) and the joint entropy (JE) between the frames. It can detect cuts, fade-ins and fade-outs. The detection technique was tested on the TRECVID2003 video test set having different types of shots and containing significant object and camera motion inside the shots. It is demonstrated that the method detects both fades and abrupt cuts with high accuracy. The information theory measure provides us with better results because it exploits the inter-frame information in a more compact way than frame subtraction. It was also successfully compared to other methods published in literature. The method for key frame extraction uses MI as well. We show that it captures satisfactorily the visual content of the shot.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/08e8ee6fd84dd21028e7f42aa68b333ce4bf13c0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Video summarisation: A conceptual framework and survey of the state of the art (2008, background): “Image features are the sole source of analysis for [7,36,41,44–49] and are all non-domain specific, seeking to identify representative key frames for inclusion in the video summary based on techniques such as assessing distortion [48] or difference levels [7,44] between frames, and determining diffe”
Video Summarization With Attention-Based Encoder–Decoder Networks (2017, background): “During these years, many approaches have been developed by exploring cues ranging from low-level visual inconsistency [10] [11], attention [3] [5] [26], to high-level semantic change of concepts [6] [13] and entities in videos [12] [34].”
A Survey on Visual Content-Based Video Indexing and Retrieval (2011, background): “1) Threshold-Based Approach: The threshold-based approach detects shot boundaries by comparing the measured pair-wise similarities between frames with a predefined threshold [47], [180]: When a similarity is less than the threshold, a boundary is detected.”
VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method (2011, background): “…has attracted considerable interest from researchers and as a result, various algorithms and techniques have been proposed in the literature, most of them based on clustering techniques (Hadi et al., 2006; Mundur et al., 2006; Chen et al., 2009; Herranz and Martinez, 2009; Furini et al., 2010).”