Reconstruction of Personalized 3D Face Rigs from Monocular Video

By Pablo Garrido, Michael Zollhöfer, Dan Casas, Levi Valgaerts, Kiran Varanasi, Patrick Pérez, Christian Theobalt. 2016

280 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

A Hybrid Model for Identity Obfuscation by Face Replacement (2018, methodology): “In contrast, parametric face models [4] give us complete control of facial attributes and have demonstrated compelling results for applications such as face reconstruction, expression transfer and dubbing [4,5,6].”
Neural Voice Puppetry: Audio-driven Facial Reenactment (2019, background): “A face model [13] is registered to the examined video and a viseme search finds video segments with similar mouth movements to the edited text.”
GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction (2019, methodology): “The most recent methods such as [36, 40, 14] use both the 3DMM model, as well as additional network structures (called correctives) in order to extend the shape and texture representation.”
GazeDirector: Fully Articulated Eye Gaze Redirection in Video (2017, background): “Garrido et al. [13] include eyeball geometry in a “detail” layer of their facial mesh.”
I M Avatar: Implicit Morphable Head Avatars from Videos (2021, methodology): “Conventional methods [15, 17, 19, 20, 41, 43, 45, 47] based on morphable mesh models [3, 27, 36] can fit the shape and texture parameters to images for a given subject.”
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction (2017, methodology): “In [14], high-quality 3D face rigs are obtained from monocular RGB video based on a multi-layer model.”
Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision (2019, background): “Until very recently, this has been the dominant paradigm [2, 30, 11].”
Learning Detailed Face Reconstruction from a Single Image (2016, methodology): “Various methods mitigate this uncertainty by using additional data such as a large photo collection of the same subject [36, 35, 21, 28, 33], continuous video frames [44, 40, 5, 11] or a rough depth map [44, 18].”
Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression (2019, background): “Face alignment plays an essential role in many face related applications such as face recognition [55, 43, 39, 70, 11], face frontalization [24, 60, 30] and 3D face reconstruction [16, 52, 37, 21].”
Deep video portraits (2018, methodology): “These combine encoderśdecoder architectures [Hinton and Salakhutdinov 2006], often with skip-connections [Ronneberger et al. 2015], with adversarial loss functions [Goodfellow et al. 2014; Radford et al. 2016].”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, methodology): “It comes close to the high-quality approach of [26], while being orders of magnitude faster and not requiring feature detection.”
Learning a model of facial shape and expression from 4D scans (2017, methodology): “Finally, images can be used to add more shape detail from shading cues as in recent work [Garrido et al. 2016].”
paGAN (2018, unknown): “Nevertheless, these dynamic face models are suitable for single-view face modeling and tracking applications [Blanz and Vetter 1999; Bouaziz et al. 2013; Cao et al. 2014, 2016; Garrido et al. 2016; Hsieh et al. 2015; Hu et al. 2017; Li et al. 2013; Saito et al. 2016, 2017; Thies et al. 2015, 2016a; ”
Adaptive 3D Face Reconstruction from Unconstrained Photo Collections (2017, background): “A video provides sufficient information for unconstrained reconstruction [27], [28], [29], [30], [31], [32], [33], [34], [70], [71] where each frame comes from the same camera and temporal coherence may be assumed.”