Personalized Cinemagraphs Using Semantic Understanding and Collaborative Learning

By Tae-Hyun Oh, Kyungdon Joo, Neel Joshi, Baoyuan Wang, In-So Kweon, Sing Bing Kang. 2017

12 citations recorded by the source corpus.

Cinemagraphs are a compelling way to convey dynamic aspects of a scene. In these media, dynamic and still elements are juxtaposed to create an artistic and narrative experience. Creating a high-quality, aesthetically pleasing cinemagraph requires isolating objects in a semantically meaningful way and then selecting good start times and looping periods for those objects to minimize visual artifacts (such a tearing). To achieve this, we present a new technique that uses object recognition and semantic segmentation as part of an optimization method to automatically create cinemagraphs from videos that are both visually appealing and semantically meaningful. Given a scene with multiple objects, there are many cinemagraphs one could create. Our method evaluates these multiple candidates and presents the best one, as determined by a model trained to predict human preferences in a collaborative way. We demonstrate the effectiveness of our approach with multiple results and a user study.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning-based Video Motion Magnification (2018, methodology): “We use the cinemagraph data generated by [21].”
Textually Customized Video Summaries (2017, background): “n retaining semantic information for better human correspondence: important objects (Lee et al. [17]), object tracks (Liu et al . [20]), motion based summaries via semantic object context (Oh et al . [26]), and object relationships (Lu et al . [21]). While these methods deal with higher level contex”