Large Scale 3D Morphable Models

By James Booth, Anastasios Roussos, Allan Ponniah, David J. Dunaway, Stefanos P. Zafeiriou. 2017

333 citations recorded by the source corpus.

We present large scale facial model (LSFM)—a 3D Morphable Model (3DMM) automatically constructed from 9663 distinct facial identities. To the best of our knowledge LSFM is the largest-scale Morphable Model ever constructed, containing statistical information from a huge variety of the human population. To build such a large model we introduce a novel fully automated and robust Morphable Model construction pipeline, informed by an evaluation of state-of-the-art dense correspondence techniques. The dataset that LSFM is trained on includes rich demographic information about each subject, allowing for the construction of not only a global 3DMM model but also models tailored for specific age, gender or ethnicity groups. We utilize the proposed model to perform age classification from 3D shape alone and to reconstruct noisy out-of-sample data in the low-dimensional model space. Furthermore, we perform a systematic analysis of the constructed 3DMM models that showcases their quality and descriptive power. The presented extensive qualitative and quantitative evaluations reveal that the proposed 3DMM achieves state-of-the-art results, outperforming existing models by a large margin. Finally, for the benefit of the research community, we make publicly available the source code of the proposed automatic 3DMM construction pipeline, as well as the constructed global 3DMM and a variety of bespoke models tailored by age, gender and ethnicity.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Generating 3D faces using Convolutional Mesh Autoencoders (2018, background): “[3] recently proposed another linear neutral expression 3D face model learned from almost 10, 000 face scans of more diverse subjects.”
Nerfies: Deformable Neural Radiance Fields (2020, background): “Domain-Specific Modeling: Many reconstruction methods use domain-specific knowledge to model the shape and appearance of categories with limited topological variation, such as faces [5, 7, 9], human bodies [30, 58], and animals [12, 64].”
StyleRig: Rigging StyleGAN for 3D Control Over Portrait Images (2020, background): “Through 3D face scanning techniques high-quality face geometry datasets can be obtained [5, 21].”
HeadNeRF: A Realtime NeRF-based Parametric Head Model (2021, methodology): “Unlike the previous 3D mesh-based topologically uniformed face parametric model [3,4,6,30,41,51,57], HeadNeRF takes the accelerated variant of neural radiance field as a unified 3D proxy, which allows it to directly control the viewing pose of the rendered result and generate high fidelity head imag”
FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction (2020, background): “Note that Table 1 doesn’t list the datasets which provide only parametric model but no source 3D models[8, 6, 34, 31].”
X2Face: A network for controlling face generation by using images, audio, and pose codes (2018, background): “fitting of 3DMMs by including high level details [34, 41], taking into account additional images [33] or 3D scans [4], or by learning 3DMM parameters directly from RGB data without ground truth labels [2, 39].”
Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset (2021, methodology): “Our work ﬁrst leverages 3DMM[1] to split the framework into two cascaded modules, named audio-to-animation module and animation-to-video module.”
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image (2019, background): “There is also an extensive literature on modelling hands [39, 53, 57, 58, 68, 69, 71, 74, 75] and faces [4, 9, 11, 13, 14, 43, 63, 76, 79] in 3D but in isolation from the rest of the body.”
InverseFaceNet: Deep Monocular Inverse Face Rendering (2017, methodology): “Recently, a large-scale parametric face model [7] was created from 10,000 facial scans, Booth et al.”
Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model Using Deep Non-Rigid Structure From Motion (2019, methodology): “In morphable models [46, 3] the mean shape and deformation basis elements are learned by PCA on a set of aligned 3D shapes, but in our case, we discover them from 2D by solving an NRSfM minimization problem that involves the projection to an unknown camera viewpoint.”
Morphable Face Models - An Open Framework (2017, methodology): “The iterative closest point algorithm (ICP) [7] and its nonrigid extension (NICP), introduced by [5] and [2] are the most popular algorithms used for establishing the correspondence of 3D face shapes ([24],[10],[3],[12],[16]).”
Learning a model of facial shape and expression from 4D scans (2017, methodology): “These measurements have previously been used to evaluate statistical models of various classes of objects, including 3D faces (e.g. [Bolkart and Wuhrer 2015; Booth et al. 2017; Brunton et al. 2014]).”
3D Morphable Face Models—Past, Present, and Future (2019, positive): “2018a; Dai et al. 2017; Paysan et al. 2009a] use this approach, combining images from multiple cameras to provide full coverage of the face with diffuse lighting to approximate albedo. A better approach is to explicitly separate shading from skin color, often referred to as intrinsic decomposition. ”