Real-time high-fidelity facial performance capture

By Chen Cao, Derek Bradley, Kun Zhou, Thabo Beeler. 2015

256 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

A Hybrid Model for Identity Obfuscation by Face Replacement (2018, neutral): “We zero-pad the image if its dimensions are smaller than the crop size, thereby obtaining the ﬁnal crop with the desired 256 × 256 size.”
Text-based editing of talking-head video (2019, methodology): “Besides model parameters, other approaches regress detailed depth maps[Richardson et al. 2017; Sela et al. 2017], or 3D displacements[Cao et al. 2015; Guo et al. 2018; Tewari et al. 2018b].”
Extreme 3D Face Reconstruction: Looking Past Occlusions (2017, methodology): “Inspired by [8], we model local deformations of the face surface as depth displacements .”
Capture, Learning, and Synthesis of 3D Speaking Styles (2019, methodology): “Cao et al. [11, 12] use a regression-based face tracker to animate the face-rig and digital avatars, while Thies et al. [55] use a landmark-based face tracker and deformation transfer [50] to reenact monocular videos.”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, methodology): “Recent methods refine a fitted prior by adding fine-scale details, either based on shape-from-shading [26, 48] or pre-learned regressors [16, 49].”
Production-level facial performance capture using deep convolutional neural networks (2016, methodology): “The early work on this area [Cao et al. 2013; Weng et al. 2014] require an actor-specific training step, but later developments have relaxed that requirement [Cao et al. 2014] and also extended the method to smaller-scale features such as wrinkles [Cao et al. 2015; Ichim et al. 2015].”
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications (2018, methodology): “Dense photometric alignment is commonly used by many approaches [GVWT13, SSS14, TZN ∗ 15, CBZB15, TZS ∗ 16a, TZS ∗ 16b, GZC ∗ 16, WBGB16].”
paGAN (2018, unknown): “Using a dedicated expression scanning session using a high-end 3D acquisition system, previous work [Cao et al. 2015] demonstrates a method that can simultaneously model a face’s overall geometry, as well as fine-scale geometry details, such as wrinkles.”
InverseFaceNet: Deep Monocular Inverse Face Rendering (2017, background): “Shading-based surface refinement can extract even fine-scale geometric surface detail [11, 19, 26, 47, 48, 52].”
Learning Detailed Face Reconstruction from a Single Image (2016, methodology): “Various methods mitigate this uncertainty by using additional data such as a large photo collection of the same subject [36, 35, 21, 28, 33], continuous video frames [44, 40, 5, 11] or a rough depth map [44, 18].”
InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image (2017, background): “Shading-based surface reﬁnement can extract even ﬁne-scale geometric surface detail [8, 14, 19, 36, 37].”
Reconstruction of Personalized 3D Face Rigs from Monocular Video (2016, methodology): “…wrinkle formation were learned from high-quality expressions (out of a vast set of examples) captured with a dense sensor array [Bermano et al. 2014; Cao et al. 2015] or with depth cameras [Li et al. 2015], or also by interpolating dense high-quality scans in a video-driven way [Fyffe et al. 2014].”
GazeDirector: Fully Articulated Eye Gaze Redirection in Video (2017, background): “However, these approaches generally avoid the eyes, cutting them out of the mesh [8, 32].”
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos (2016, background): “Follow-up work [4] also regresses fine-scale face wrinkles.”
A deep learning approach for generalized speech animation (2017, unknown): “The main alternative to hand animation is performance-driven animation using facial motion capture of an actor’s face [Beeler et al. 2011; Cao et al. 2015, 2013; Fyffe et al. 2014; Huang et al. 2011; Li et al. 2013; Weise et al. 2011; Weng et al. 2014; Zhang et al. 2004].”
Dynamic FAUST: Registering Human Bodies in Motion (2017, methodology): “Texture has been used for aligning isolated body parts such as faces and hands [6, 5, 17, 23, 43].”
Adaptive 3D Face Reconstruction from Unconstrained Photo Collections (2017, background): “Dense correspondence among images is the basis for multi-view stereo and photometric stereo reconstruction approaches.”