FaceWarehouse: A 3D Facial Expression Database for Visual Computing

By Chen Cao, Yanlin Weng, Shun Zhou, Y. Tong, Kun Zhou. 2014

1,170 citations recorded by the source corpus.

We present FaceWarehouse, a database of 3D facial expressions for visual computing applications. We use Kinect, an off-the-shelf RGBD camera, to capture 150 individuals aged 7-80 from various ethnic backgrounds. For each person, we captured the RGBD data of her different expressions, including the neutral expression and 19 other expressions such as mouth-opening, smile, kiss, etc. For every RGBD raw data record, a set of facial feature points on the color image such as eye corners, mouth contour, and the nose tip are automatically localized, and manually adjusted if better accuracy is required. We then deform a template facial mesh to fit the depth data as closely as possible while matching the feature points on the color image to their corresponding points on the mesh. Starting from these fitted face meshes, we construct a set of individual-specific expression blendshapes for each person. These meshes with consistent topology are assembled as a rank-3 tensor to build a bilinear face model with two attributes: identity and expression. Compared with previous 3D facial databases, for every person in our database, there is a much richer matching collection of expressions, enabling depiction of most human facial actions. We demonstrate the potential of FaceWarehouse for visual computing with four applications: facial image manipulation, face component transfer, real-time performance-based facial image animation, and facial animation retargeting from video to image.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

A Hybrid Model for Identity Obfuscation by Face Replacement (2018, methodology): “high-quality face scans [10]. are the coecients of the expression basis vectors computed using PCA on selected blend shapes of [29] A Hybrid Model for Identity Obfuscation by Face Replacement 5 and [30]. We use 80 , 80 and 64 parameters. Together, they dene the pervertex position and re ectance of t”
Generating 3D faces using Convolutional Mesh Autoencoders (2018, background): “Existing large scale datasets [14, 16, 50, 49, 38] do not contain high resolution extreme facial expressions.”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “model is generated by blending models of the individual component meshes: SMPL [25] for the body, FaceWarehouse [31] for the face, and an artist rigged for the hand.”
InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image (2017, methodology): “We then show qualitative and quantitative results and comparisons on the datasets CelebA [29], LFW (Labeled Faces in the Wild) [16], FaceWarehouse [7], Thomas [13], Volker [47] and 300-VW (300 Videos in the Wild) [40].”
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild (2020, positive): “By contrast, MFN [2] employs FacewareHouse [1] as the training data to improve the 3D fitting under exaggerated expressions.”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, methodology): “The subspace of expression variations is spanned by the vectors {bgk} ms+me k=ms+1 which were created using PCA from a subset of blendshapes of [2] and [18].”
High-fidelity Pose and Expression Normalization for face recognition in the wild (2015, methodology): “The identity shape Aid comes from the Basel Face Model (BFM) [36] and the expression Aexp comes from the Face Warehouse [14].”
Deep Kinematic Pose Regression (2016, background): “[16] represent 3D face landmarks by a linear combination of shape bases [23] and expression bases [4].”
DurIAN: Duration Informed Attention Network For Multimodal Synthesis (2019, methodology): “were extracted from the regression to the Principle Component 8 DurIAN: Duration Informed Attention Network For Multimodal Synthesis Analysis(PCA) leading factors of face RGB features as described in [26]. And the last 7 dimensional features record the 3D position of current facial expression. Durin”
FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction (2020, methodology): “The ﬁrst direction is to separate the parametric space to multiple dimensions like identity, expression and visemes, so that the model could be controlled by these attributes separately[52, 12, 30, 27].”
Reconstruction of Personalized 3D Face Rigs from Monocular Video (2016, methodology): “2009] and the FaceWarehouse database [Cao et al. 2014b]:”
Modeling Facial Geometry Using Compositional VAEs (2018, background): “However, none of the publicly available face shape datasets [6, 5] offer truly high-resolution models, which would not allow us to fully test descriptive power of our compositional model.”
paGAN (2018, neutral): “…these dynamic face models are suitable for single-view face modeling and tracking applications [Blanz and Vetter 1999; Bouaziz et al. 2013; Cao et al. 2014, 2016; Garrido et al. 2016; Hsieh et al. 2015; Hu et al. 2017; Li et al. 2013; Saito et al. 2016, 2017; Thies et al. 2015, 2016a;…”
Learning a model of facial shape and expression from 4D scans (2017, methodology): “More recently, the FaceWarehouse model [Cao et al. 2014] uses scans of 150 people with variation in age and ethnicity and with 20 different facial poses.”
GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models (2020, methodology): “The Frank model[15] is based on a simplified version of the SMPL body[24], to which it connects an artist-designed hand rig, and the public FaceWarehouse head[8].”
Multilinear Autoencoder for 3D Face Model Learning (2018, methodology): “They were later used to edit 2D images and videos with the help of 3D face reconstructions [10, 6].”