Video Primal Sketch: A Unified Middle-Level Representation for Video
By Zhi Han, Zongben Xu, Song-Chun Zhu. 2015
7 citations recorded by the source corpus.
This paper presents a middle-level video representation named video primal sketch (VPS), which integrates two regimes of models: (i) sparse coding model using static or moving primitives to explicitly represent moving corners, lines, feature points, etc., (ii) FRAME /MRF model reproducing feature statistics extracted from input video to implicitly represent textured motion, such as water and fire. The feature statistics include histograms of spatio-temporal filters and velocity distributions. This paper makes three contributions to the literature: (i) Learning a dictionary of video primitives using parametric generative models; (ii) Proposing the spatio-temporal FRAME and motion-appearance FRAME models for modeling and synthesizing textured motion; and (iii) Developing a parsimonious hybrid model for generic video representation. Given an input video, VPS selects the proper models automatically for different motion patterns and is compatible with high-level action representations. In the experiments, we synthesize a number of textured motion; reconstruct real videos using the VPS; report a series of human perception experiments to verify the quality of reconstructed videos; demonstrate how the VPS changes over the scale transition in videos; and present the close connection between VPS and high-level action models.
Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Song-Chun Zhu
- Ying Nian Wu
- Jianwen Xie
- Erik Nijkamp
- Luc Van Gool
- Mitch Hill
- Alan L. Yuille
- Zhuowen Tu
In Control and Systems Engineering
Citations in the corpus
- Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, background): “Generating dynamic textures or textured motions have been studied by [1], [2], [41], [42].”
- Synthesizing Dynamic Patterns by Spatial-Temporal Generative ConvNet (2016, background): “Dynamic textures or textured motions have been studied by [2, 24, 25, 9].”