3D Face Morphable Models "In-the-Wild"

By James Booth, Epameinondas Antonakos, Stylianos Ploumpis, George Trigeorgis, Yannis Panagakis, Stefanos P. Zafeiriou. 2017

192 citations recorded by the source corpus.

3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions (in-the-wild). In this paper, we propose the first, to the best of our knowledge, in-the-wild 3DMM by combining a powerful statistical model of facial shape, which describes both identity and expression, with an in-the-wild texture model. We show that the employment of such an in-the-wild texture model greatly simplifies the fitting procedure, because there is no need to optimise with regards to the illumination parameters. Furthermore, we propose a new fast algorithm for fitting the 3DMM in arbitrary images. Finally, we have captured the first 3D facial database with relatively unconstrained conditions and report quantitative evaluations with state-of-the-art performance. Complementary qualitative reconstruction results are demonstrated on standard in-the-wild facial databases.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

A Hybrid Model for Identity Obfuscation by Face Replacement (2018, background): “Higher-quality models have been constructed using more scans [11], or by using information from in-the-wild images [12,13].”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, methodology): “[10], the face model is augmented with an ‘in-the-wild’ texture model.”
Extreme 3D Face Reconstruction: Seeing Through Occlusions (2017, methodology): “Deep methods were also recently proposed for this purpose [6, 11, 24, 49, 51].”
InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image (2017, methodology): “We describe the face model we use in Section 4.”
Extreme 3D Face Reconstruction: Looking Past Occlusions (2017, methodology): “Deep methods were also recently proposed for this purpose [7, 11, 23, 45, 46].”
InverseFaceNet: Deep Monocular Inverse Face Rendering (2017, background): “[6] extend 3D morphable models to “in-the-wild” conditions, and deep appearance models [17] extend active appearance models by capturing geometry and appearance of faces more accurately under large unseen variations.”
FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction (2020, background): “Model fitting datasets[33, 60, 23, 5, 7] fit the 3D morphable model to the collected images, which makes it convenient to build a large-scale dataset on the base of wild faces.”
GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction (2019, methodology): “Furthermore, with appropriate statistical texture models on image features such as Scale Invariant Feature Transform (SIFT) and Histogram Of Gradients (HOG), 3DMMbased methodologies can still achieve state-of-the-art performance in 3D shape estimation on images captured under unconstrained condition”
Nonlinear 3D Face Morphable Model (2018, background): “[7] aim to improve texture of 3DMM to go beyond controlled settings by learning inthe-wild feature-based texture model.”