Displaced dynamic expression regression for real-time facial tracking and animation

By Chen Cao, Qiming Hou, Kun Zhou. ACM Transactions on Graphics, 2014

409 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/75ed53365aace52cae1128aacc84b51536fe10ca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

First Order Motion Model for Image Animation (2020, methodology): “3D model) [4] and resorting to computer graphics techniques [6, 33].”
Real-time expression transfer for facial reenactment (2015, methodology): “Comparison to Cao et al. 2014 We also compare our method to the real-time face tracker of Cao et al. [2014a], which tracks 2D facial landmarks and infers the 3D face shape from a single RGB video stream.”
Recycle-GAN: Unsupervised Video Retargeting (2018, background): “The ﬁrst set is speciﬁcally designed for domains such as human faces [5,41,42].”
FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction (2020, methodology): “There are many existing performance driven facial animation methods generating blendshape weights with depth camera[51, 28, 9] or single RGB camera[11, 10, 14].”
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications (2018, methodology): “Given a monocular video as input, parameter estimation is commonly phrased as a general non-linear optimization problem [LYYB13,BWP13,CWS ∗ 13,CWLZ13,SWTC14,SSS14,CHZ14, GZC ∗ 16,TZN ∗ 15,TZS ∗ 16a,WSXC16,TZS ∗ 16b].”
GazeDirector: Fully Articulated Eye Gaze Redirection in Video (2017, background): “Such models can then be ﬁt to either RGB-D data [33, 38] or RGB data [7, 31, 32].”
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network (2018, methodology): “Most of earlier methods are to establish the correspondences of the special points between input images and the 3D mean geometry including landmarks[21,6,22,7,23,24,25] and local features[26,27,25], then solve the non-linear optimization function to regress the 3DMM coeﬃcients.”
Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set (2019, methodology): “Beyond 3DMM, another popular 3D face model is the multilinear tensor model [54, 10, 9, 43].”
Reconstruction of Personalized 3D Face Rigs from Monocular Video (2016, methodology): “However, generic blendshape models and identity models alone [Cao et al. 2014a; Garrido et al. 2013; Shi et al. 2014] fail to capture important person-specific expression and identity details learned by our approach.”
3D Morphable Face Models—Past, Present, and Future (2019, positive): “Many methods can track the face in real-time [Cao et al. 2015, 2014a, 2013, 2016a; Ichim et al. 2015; Thies et al. 2016].”
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos (2016, background): “2014 [7]: They capture face performance from monocular RGB in real-time.”
Capture, Learning, and Synthesis of 3D Speaking Styles (2019, methodology): “[11, 12] use a regression-based face tracker to animate the face-rig and digital avatars, while Thies et al.”
Motion Representations for Articulated Animation (2021, background): “Such knowledge typically includes landmarks [4, 40, 24, 12], semantic segmentations [22], and parametric 3D models [11, 32, 9, 20, 18].”
High-fidelity Pose and Expression Normalization for face recognition in the wild (2015, methodology): “An alternative method is the landmark based 3D face model fitting [5, 1, 46, 13, 12, 38], which estimates the model parameters with the correspondence between 2D and 3D landmarks.”
Animating Arbitrary Objects via Deep Motion Transfer (2018, methodology): “Traditional approaches [7, 38] are designed for specific domains, as they operate only on faces, human silhouettes, etc.”
Effective face frontalization in unconstrained images (2014, neutral): “These and other concerns have been suggested in the past as reasons for preferring other approaches to pose estimation [3, 24].”
Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting (2016, methodology): “Thus, at the stage k of the cascaded CNN, we can learn a CNN to estimate the desired update of the projection parameter, Θkm = argmin Θkm Nd∑ i=1 ||∆mki −CNNkm(Ii,Ui,vk−1i ; Θ k m)||2, (6) where the true projection update is the difference between the current projection parameter and the ground…”
MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model (2023, neutral): “Conversely, explicit methods warp the source image to the target by either 2D optical flow [22, 26–28, 46, 52], 3D deformation field [3, 17, 36], or directly sawpping the face of target image [18].”