Example-based facial rigging

By Hao Li, Thibaut Weise, Mark Pauly. 2010

252 citations recorded by the source corpus.

We introduce a method for generating facial blendshape rigs from a set of example poses of a CG character. Our system transfers controller semantics and expression dynamics from a generic template to the target blendshape model, while solving for an optimal reproduction of the training poses. This enables a scalable design process, where the user can iteratively add more training poses to refine the blendshape expression space. However, plausible animations can be obtained even with a single training pose. We show how formulating the optimization in gradient space yields superior results as compared to a direct optimization on blendshape vertices. We provide examples for both hand-crafted characters and 3D scans of a real actor and demonstrate the performance of our system in the context of markerless art-directable facial tracking.

Subfields: Computer Vision and Pattern Recognition, Dermatology, Developmental Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Dermatology

In Developmental Biology

Citations in the corpus

Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid (2013, background): “[117], physical models relating tissues, bones and muscles”
Online modeling for realtime facial animation (2013, methodology): “For a given neutral expression b0 we define approximations for all other blendshapes using a variant of deformation transfer [Sumner and Popović 2004], see also [Li et al. 2010].”
Realtime facial animation with on-the-fly correctives (2013, methodology): “To improve deployability, data-driven techniques are often used to reduce the amount of pre-processing at the cost of synthesized reconstructions from priors that may look believable but not necessarily capture the characteristics of the spe-ciﬁc actor [Li et al. 2010; Weise et al. 2011].”
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications (2018, methodology): “In this category, we can ﬁnd methods that either adapt [LWP10, IBP15] or extend [LYYB13] the linear expression”
Learning a model of facial shape and expression from 4D scans (2017, methodology): “Such personalized rigs typically require a userspecific calibration or training procedure [Li et al. 2010; Weise et al. 2011].”
3D shape regression for real-time facial animation (2013, methodology): “Since the setup images consist of a pre-defined standard expressions, their blendshape expression coefficients should be consistent with standard blendshape coefficients a for these expressions [Li et al. 2010].”
Production-level facial performance capture using deep convolutional neural networks (2016, methodology): “Since the comparison methods generalize to any identity and our method assumes the identity of one user, in order to make the comparison more fair, we use the method of Li et al. [2010] to fix the identity mesh for the comparison methods and only track the expressions.”
FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction (2020, methodology): “The models in expression dimension could be further transformed to a set of blendshapes[28], which can be rigged to generate individual-speciﬁc animation.”
3D Morphable Face Models—Past, Present, and Future (2019, unknown): “Li et al. [2010] propose a method to adapt a pre-defined blendshape model to a specific subject given a small number of static face scans in different expressions, which provides a personalized facial rig.”
Capture, Learning, and Synthesis of 3D Speaking Styles (2019, methodology): “Li et al. [36] and Weise et al. [60] capture example-based rigs in an ofﬂine calibration procedure to build personalized face-rigs, Bouaziz et al. [6] use a generic identity model.”
Total Moving Face Reconstruction (2014, background): “Instead of requiring the person to be scanned in the lab or participate in the reconstruction process (as many other methods require [24,18,10,41,33]), we leverage whatever existing imagery is available online or in personal photo collections.”
Review of statistical shape spaces for 3D data with comparative analysis for human faces (2012, methodology): “An example of such an artist-generated basis are blendshape models (as for instance used by Li et al. [2]), which can be used to encode a face performing different expressions.”
paGAN (2018, unknown): “Existing methods that can capture facial expression and appearance variations rely on multiple input images which correspond to a set of key expressions [Cao et al. 2014, 2016; Casas et al. 2016; Li et al. 2010; Weise et al. 2011, 2009].”