Internet Based Morphable Model

By Ira Kemelmacher-Shlizerman. 2013

73 citations recorded by the source corpus.

In this paper we present a new concept of building a morphable model directly from photos on the Internet. Morphable models have shown very impressive results more than a decade ago, and could potentially have a huge impact on all aspects of face modeling and recognition. One of the challenges, however, is to capture and register 3D laser scans of large number of people and facial expressions. Nowadays, there are enormous amounts of face photos on the Internet, large portion of which has semantic labels. We propose a framework to build a morph able model directly from photos, the framework includes dense registration of Internet photos, as well as, new single view shape reconstruction and modification algorithms.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Category-specific object reconstruction from a single image (2014, background): “[23, 32] which aims to learn morphable models for faces from 2D images, but we focus on richer shapes in unconstrained settings, at the expense of lower resolution reconstructions.”
Adaptive 3D Face Reconstruction from Unconstrained Photo Collections (2017, background): “Photo collections are used for a variety of non-reconstruction purposes such as improved fitting of a 3DMM [37] and even creating a 3DMM without range scans [12], [38].”
A 3D Morphable Model Learnt from 10,000 Faces (2016, methodology): “Kemelmacher recently presented a technique that attempts to learn a full 3D facial model automatically from thousands of images [21].”
Text-based editing of talking-head video (2019, methodology): “There is a large body of work on reconstructing facial geometry and appearance from a single image using optimization methods[Fyffe et al. 2014; Garrido et al. 2016; Ichim et al. 2015; Kemelmacher-Shlizerman 2013; Roth et al. 2017; Shi et al. 2014; Suwajanakorn et al. 2017; Thies et al. 2016].”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, background): “, photo collections [36], internet photos [35] or videos [59].”
Total Moving Face Reconstruction (2014, background): “Most related to our work are single view methods, particularly [30,28,27,29].”
Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model Using Deep Non-Rigid Structure From Motion (2019, background): “egories from keypoints and masks [7, 18], incorporating spatio-temporal priors [38] and illumination models [27], while leading to impressively high-resolution 3D Reconstruction results [14, 27, 16, 22, 24, 23].”
Large Scale 3D Morphable Models (2017, background): “done in Patel and Smith (2009). – Optical flow (OF) estimation, as e.”
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications (2018, methodology): “Alternative parametric face models were built later, using inputs ranging from commodity [CFKP04] monocular face images downloaded from the internet was also investigated [KS13].”
Photorealistic Facial Texture Inference Using Deep Neural Networks (2016, background): “Several extensions have been proposed leveraging internet images [24] and large-scale 3D facial scans [6].”
3D Face Morphable Models "In-the-Wild" (2017, methodology): “ITW slightly outperforms IMM even though both IMM and PS-NL use all four available images of each subject.”