Towards Fast, Accurate and Stable 3D Dense Face Alignment

By Jianzhu Guo, Xiangyu Zhu, Yang Yang, Zhen Lei, S. Li. European Conference on Computer Vision, 2020

500 citations recorded by the source corpus.

Existing methods of 3D dense face alignment mainly concentrate on accuracy, thus limiting the scope of their practical applications. In this paper, we propose a novel regression framework named 3DDFA-V2 which makes a balance among speed, accuracy and stability. Firstly, on the basis of a lightweight backbone, we propose a meta-joint optimization strategy to dynamically regress a small set of 3DMM parameters, which greatly enhances speed and accuracy simultaneously. To further improve the stability on videos, we present a virtual synthesis method to transform one still image to a short-video which incorporates in-plane and out-of-plane face moving. On the premise of high accuracy and stability, 3DDFA-V2 runs at over 50fps on a single CPU core and outperforms other state-of-the-art heavy models simultaneously. Experiments on several challenging datasets validate the efficiency of our method. Pre-trained models and code are available at this https URL.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/a209207236b3a08bb01a59f1084bcbb98a7e8ee1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Learning an animatable detailed 3D face model from in-the-wild images (2020, methodology): “We compare DECA with publicly available methods, namely 3DDFA-V2 [Guo et al. 2020], Deng et al. [2019], RingNet [Sanyal et al. 2019], PRNet [Feng et al. 2018b], 3DMM-CNN [Tran et al. 2017] and Ex-treme3D [Tran et al. 2018].”
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild (2019, methodology): “For objects such as human bodies and human faces, some methods [16], [17], [18], [20], [21], [29], [57], [58], [59], [60], [61] have learn to reconstruct from raw images, but starting from the knowledge of a predefined shape model, such as SMPL [62] or Basel [11], or shape templates.”
High-Fidelity GAN Inversion for Image Attribute Editing (2021, background): “A naive alternative is to directly add the distortion map ∆̃ to X o in the image space with warping estimated by face landmarks [12].”
Video Transformer for Deepfake Detection with Incremental Learning (2021, methodology): “To extract more informative features, we train our models on the aligned facial images and their corresponding UV texture maps [16, 17, 26].”
Face Anti-Spoofing with Human Material Perception (2020, methodology): “Dense face alignment [60,61] is adopted to estimate facial 3D shapes and generate the facial depth maps with size 32× 32.”
EMOCA: Emotion Driven Monocular Face Capture and Animation (2022, methodology): “The latter are trained fully supervised [15, 34, 42, 67, 87, 88, 98] or self-supervised with predicted 2D keypoints [19, 27, 52, 73, 76, 80, 82, 83, 96], 2D face contours [52], photometric constraints [19, 27, 30, 76, 80, 82, 83, 96], face recognition features [19, 27, 30, 76], multi-view constraint”