Morphable Face Models - An Open Framework

By Thomas Gerig, Andreas Forster, Clemens Blumer, B. Egger, Marcel Lüthi, Sandro Schönborn, Thomas Vetter. IEEE International Conference on Automatic Face & Gesture Recognition, 2017

312 citations recorded by the source corpus.

In this paper, we present a novel open-source pipeline for face registration based on Gaussian processes as well as an application to face image analysis. Non-rigid registration of faces is significant for many applications in computer vision, such as the construction of 3D Morphable face models (3DMMs). Gaussian Process Morphable Models (GPMMs) unify a variety of non-rigid deformation models with B-splines and PCA models as examples. GPMM separate problem specific requirements from the registration algorithm by incorporating domain-specific adaptions as a prior model. The novelties of this paper are the following: (i) We present a strategy and modeling technique for face registration that considers symmetry, multi-scale and spatially-varying details. The registration is applied to neutral faces and facial expressions. (ii) We release an open-source software framework for registration model-building demonstrated on the publicly available BU3D-FE database. The released pipeline also contains an implementation of an Analysis-by-Synthesis model adaption of 2D face images, tested on the Multi-PIE and LFW database. This enables the community to reproduce, evaluate and compare the individual steps of registration to model-building and 3D/2D model fitting. (iii) Along with the framework release, we publish a new version of the Basel Face Model (BFM-2017) with an improved age distribution and an additional facial expression model.

View Paper: https://www.semanticscholar.org/paper/d57e332c84671806219a29631c2945a58faf2702

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

3D Morphable Face Models—Past, Present, and Future (2019, unknown): “[Gerig et al. 2018] establish dense correspondence with a Gaussian process deformation model with the spatially varying kernel.”
Unsupervised Training for 3D Morphable Model Regression (2018, methodology): “We map features from a facial recognition network [25] into identity parameters for the Basel 2017 Morphable Face Model [8]. to-image autoencoder with a ﬁxed, morphable-model-based decoder and an image-based loss [28].”
Fake it till you make it: face analysis in the wild using synthetic data alone (2021, methodology): “In contrast to other work which approximates hair with textures or coarse geometry [17, 55], we represent hair as individual 3D strands, with a full head of hair comprising over 100,000 strands.”
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild (2019, methodology): “For objects such as human bodies and human faces, some methods [26, 17, 60, 14] have learn to reconstruct from raw images, but starting from the knowledge of a predefined shape model such as SMPL [34] or Basel [43].”
Deformable Shape Completion with Graph Convolutional Autoencoders (2017, methodology): “A viable alternative is model-based approaches, where a parametric morphable model describing the variability of a certain class of objects can be fit to the observed data [4, 16].”
Synthetic Data for Deep Learning (2019, methodology): “They use the Basel Face Model 2017 [199], a 3D morphable model of face shape [61,63], and take special care to randomize the pose, camera location, illumination conditions, and background.”
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era (2019, background): “Gerig et al. [124] extended the model by including expressions as a separate space.”
Learning an animatable detailed 3D face model from in-the-wild images (2020, methodology): “Such methods can be categorized into optimization-based [Aldrian and Smith 2013; Bas et al. 2017; Blanz et al. 2002; Blanz and Vetter 1999; Gerig et al. 2018; Romdhani and Vetter 2005; Thies et al. 2016], or learning-based methods [Chang et al. 2018; Deng et al. 2019; Genova et al. 2018; Kim et al.…”
Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs (2019, methodology): “A 3DMM, such as BFM2017, the 2017 edition of the Basel Face Model [7], is usually obtained through principal component analysis.”