Learning a model of facial shape and expression from 4D scans

By Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, Javier Romero. 2017

1,687 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Generating 3D faces using Convolutional Mesh Autoencoders (2018, methodology): “Several face models use PCA space to represent identity and expression variations [41, 28, 1, 7, 47].”
InverseFaceNet: Deep Monocular Inverse Face Rendering (2017, background): “These seminal face models are powerful and expressive, and remain useful for many applications even though more complex and accurate appearance models exist [30, 37].”
Neural Voice Puppetry: Audio-driven Facial Reenactment (2019, methodology): “From this embedding, VOCA regresses 3D vertices on a FLAME face model [21] conditioned on a subject label.”
Recent advances in 3D Gaussian splatting (2024, neutral): “For easier head expression controllability, GaussianAvatars [145] introduce the geometric priors (Flame parametric face model [143]) into 3DGS, which binds the Gaussians onto the explicit mesh and optimize the parameters of Gaussian ellipsoids.”
HeadNeRF: A Realtime NeRF-based Parametric Head Model (2021, methodology): “Early works of the parametric face/head model [3, 6, 11, 30, 41, 57] mainly model 3D faces with the topologically uniformed face template mesh and usually ignore to represent the non-face parts, such as hair and teeth.”
Multilinear Autoencoder for 3D Face Model Learning (2018, methodology): “[18] recently introduced a generative model learned from a large collection of 3D motion sequences of faces.”
Capture, Learning, and Synthesis of 3D Speaking Styles (2019, methodology): “[37] capture facial performance with a high-resolution scanner and animate static face meshes using an articulated generic head model.”
Fake it till you make it: face analysis in the wild using synthetic data alone (2021, methodology): “It is a blendshape-based face rig similar to previous work [17, 34], and comprises a mesh of N=7, 667 Figure 3.”
AMASS: Archive of Motion Capture As Surface Shapes (2019, methodology): “will extend SSM dataset to include captures with articulated hands. We also intend to extend MoSh++ to work with facial mocap markers. This should be possible using the recently published FLAME model [25], which uses a similar parametrization as the SMPL and MANO models to model facial identity and ”
GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (2023, methodology): “INSTA [55] directly warps points according the nearest FLAME [21] mesh tri-angle.”
Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision (2019, methodology): “Here we use FLAME [21] because it is more accurate than other models, captures a wide range of shapes, models the whole head and neck, can be easily animated, and is freely available.”
3D Morphable Face Models—Past, Present, and Future (2019, neutral): “First, off-line techniques have been proposed [Blanz et al. 2003; Bregler et al. 1997; Kemelmacher-Shlizerman et al. 2010; Li et al. 2014a, 2012; Theobald et al. 2009; Vlasic et al. 2005b].”
GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models (2020, methodology): “In turn, SMPL-X attaches the FLAME[21] head to the SMPLH (body and hand) model[35] and refits it to an additional set of 5,586 scans.”
AI-generated characters for supporting personalized learning and well-being (2021, methodology): “The following models are used: VOCA 63 (Voice Operated Character Animation), FLAME 64 (Faces Learned with an Articulated Model and Expressions), Speech-Driven Facial Animation 65 and First Order Motion Model 66 .”
Learning an animatable detailed 3D face model from in-the-wild images (2020, methodology): “Given a dataset of 2 𝐷 face images 𝐼 𝑖 with multiple images per subject, corresponding identity labels 𝑐 𝑖 , and 68 2 𝐷 keypoints k 𝑖 per image, the coarse reconstruction branch is trained by minimizing Eye closure loss: The eye closure loss computes the relative offset of landmarks k 𝑖 and ”
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image (2019, methodology): “We combine SMPL with the FLAME head model [43] and the MANO hand model [67] and then register this combined model to 5586 3D scans that we curate for quality.”
FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction (2020, methodology): “We compare our model to FaceWarehouse(FW)[12] and FLAME[30] by ﬁtting them to our self-captured test set, which consists of 1000 high quality meshes from 50 subjects performing 20 different expressions each.”
FastNeRF: High-Fidelity Neural Rendering at 200FPS (2021, methodology): “We then fit a 3D face model similar to FLAME [13] to obtain expression parameters for each frame.”
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications (2018, methodology): “Recently, the FLAME shape and expression model [LBB ∗ 17] has been released which has been constructed from thousands of 4D scans.”
I M Avatar: Implicit Morphable Head Avatars from Videos (2021, methodology): “5, we demonstrate control over expressions and poses by inter- and extrapolating an example expression (first expression component in FLAME [27]), and jaw (pitch) and neck (yaw) poses separately.”