A system for retargeting of streaming video

By Philipp Krahenbuhl, Manuel Lang, Alexander Sorkine-Hornung, Markus H. Gross. 2009

274 citations recorded by the source corpus.

We present a novel, integrated system for content-aware video retargeting. A simple and interactive framework combines key frame based constraint editing with numerous automatic algorithms for video analysis. This combination gives content producers high level control of the retargeting process. The central component of our framework is a non-uniform, pixel-accurate warp to the target resolution which considers automatic as well as interactively defined features. Automatic features comprise video saliency, edge preservation at the pixel resolution, and scene cut detection to enforce bilateral temporal coherence. Additional high level constraints can be added by the producer to guarantee a consistent scene composition across arbitrary output formats. For high quality video display we adopted a 2D version of EWA splatting eliminating aliasing artifacts known from previous work. Our method seamlessly integrates into postproduction and computes the reformatting in real-time. This allows us to retarget annotated video streams at a high quality to arbitary aspect ratios while retaining the intended cinematographic scene composition. For evaluation we conducted a user study which revealed a strong viewer preference for our method.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Generative Visual Manipulation on the Natural Image Manifold (2016, methodology): “More advanced editing methods such as image warping [8,9] or structured image editing [10] intelligently reshuffle the pixels in an image following user’s edits.”
A comparative study of image retargeting (2010, unknown): “…Seam-Carving ( SC ) [Rubinstein et al. 2008], Scale-and-Stretch ( SNS ) [Wang et al. 2008], Multi-operator ( MULTIOP ) [Ru-binstein et al. 2009], Shift-maps ( SM ) [Pritch et al. 2009], Streaming Video ( SV ) [Kr¨ahenb¨uhl et al. 2009], and Energy-based deformation ( LG ) [Karni et al. 2009] 1 .”
Nonlinear disparity mapping for stereoscopic 3D (2010, methodology): “In addition to automatic constraints it would be interesting to include the possibility to manually add high level constraints regarding region positions or global lines [Kr¨ahenb¨uhl et al. 2009].”
Light Field Image Processing: An Overview (2017, unknown): “This structure is the key difference with other methods targeting non-structured media, such as unstructured light fields [217], [218], video [209], [219], [220], [221] or image collections [222], [223], [224].”
Making 360° Video Watchable in 2D: Learning Videography for Click Free Viewing (2017, background): “Video retargeting Video retargeting adapts a source video by cropping and scaling to better fit the target display while minimizing the information loss [5,22,23,26,29,34].”