Exploring photobios

By Ira Kemelmacher-Shlizerman, Eli Shechtman, Rahul Garg, S. M. Seitz. 2011

91 citations recorded by the source corpus.

We present an approach for generating face animations from large image collections of the same person. Such collections, which we call photobios, sample the appearance of a person over changes in pose, facial expression, hairstyle, age, and other variations. By optimizing the order in which images are displayed and cross-dissolving between them, we control the motion through face space and create compelling animations (e.g., render a smooth transition from frowning to smiling). Used in this context, the cross dissolve produces a very strong motion effect; a key contribution of the paper is to explain this effect and analyze its operating range. The approach operates by creating a graph with faces as nodes, and similarities as edges, and solving for walks and shortest paths on this graph. The processing pipeline involves face detection, locating fiducials (eyes/nose/mouth), solving for pose, warping to frontal views, and image comparison based on Local Binary Patterns. We demonstrate results on a variety of datasets including time-lapse photography, personal photo collections, and images of celebrities downloaded from the Internet. Our approach is the basis for the Face Movies feature in Google's Picasa.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Generative Visual Manipulation on the Natural Image Manifold (2016, background): “This is often done by constructing an image graph with images as nodes, and pairwise distance function as the edge, and computing a shortest path between the starting image and end image [13].”
FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences (2015, background): “Recent work exploits the connectivity within an image collection by deﬁning a graph over images (e.g. [15, 27, 47, 13]) or objects (e.g. [28, 20, 6]).”
Deep Feature Interpolation for Image Content Changes (2016, background): “In concept, our work is similar to [41, 10, 42, 19, 17] that use video or photo collections to transfer the personality and character of one person’s face to a different person (a form of puppetry [39, 45, 20]).”
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos (2016, methodology): “tor. Recent work presents virtual dubbing [14], a problem similar to ours; however, the method runs at slow ofﬂine rates and relies on a generic teeth proxy for the mouth interior. Kemelmacher et al. [20] generate face animations from large image collections, but the obtained results lack temporal c”
Deep video portraits (2018, background): “…the inner face region, to single images [Blanz et al. 2004; Blanz and Vetter 1999], unstructured image collections [KemelmacherShlizerman 2013; Kemelmacher-Shlizerman et al. 2011; Roth et al. 2017] or video [Cao et al. 2014b; Fyfe et al. 2014; Garrido et al. 2016; Ichim et al. 2015; Shi et…”
A survey on face detection in the wild: Past, present and future (2015, background): “…[2], facial behaviour analysis [3], facial attribute recognition [4] (i.e., gender/age recognition [5] and assessment of beauty [6]), face relighting and morphing [7], facial shape reconstruction [8], image and video retrieval, as well as organization and presentation of digital photo-albums [9].”
Real-time expression transfer for facial reenactment (2015, background): “If the task is to create a new facial animation, additional temporal coherence constraints must be embedded in the objective to minimize possible in-between jumps along the sequence [Kemelmacher-Shlizerman et al. 2011].”