Real-time expression transfer for facial reenactment

By Justus Thies, Michael Zollhöfer, Matthias Nießner, Levi Valgaerts, Marc Stamminger, Christian Theobalt. 2015

404 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Generating 3D faces using Convolutional Mesh Autoencoders (2018, background): “The existing state of the art 3D face representations mostly use linear transformations [41, 28, 42] or higher-order tensor generalizations [46, 12, 14].”
Learning to Navigate the Energy Landscape (2016, methodology): “Recent years have seen the re-emergence of the ‘analysis by synthesis’ or ’inverse graphics’ approach to Computer Vision problems [3, 42, 39, 27, 20, 43].”
GazeDirector: Fully Articulated Eye Gaze Redirection in Video (2017, background): “Such models can then be ﬁt to either RGB-D data [33, 38] or RGB data [7, 31, 32].”
Adaptive 3D Face Reconstruction from Unconstrained Photo Collections (2017, background): “Dense correspondence among images is the basis for multi-view stereo and photometric stereo reconstruction approaches.”
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing (2020, methodology): “Moreover, they focus on transferring the expressions without the head movement [65, 69, 70, 71, 73].”
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models (2019, methodology): “y accuracy from other hand-crafted metrics which average over training dataset. Therefore, GANs have shown improved photorealism in image synthesis [17,54,92], translation [45,93,94], or manipulation [8,79,80]. On the other hand, because of the nature of excluding sample-wise supervision, GANs impli”
Cross-Domain and Disentangled Face Manipulation With 3D Guidance (2021, background): “manipulation applications [33], [34], [35], [36], [37], but they only capture the coarse geometry due to the mostly linear statistical model.”
Reconstruction of Personalized 3D Face Rigs from Monocular Video (2016, neutral): “We believe that a drastic reduction of the computation time is possible by harnessing the data-parallel processing power of modern GPUs, as recently demonstrated for nonlinear optimization [Thies et al. 2015; Wu et al. 2014; Zollhöfer et al. 2014].”
Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints (2018, background): “The adversarial framework of GANs can also be used in conditional scenarios for image translation [36, 70, 71] or manipulation in a given context [60, 61, 57, 12, 64], which diversifies media synthesis.”
Learning a model of facial shape and expression from 4D scans (2017, methodology): “The recently published Face2Face framework [Thies et al. 2015] uses a similar model combining linear identity and expression models with an additional linear albedo model to capture appearance.”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “Face2Face [38] is an advanced form of facial reenactment technique, as presented in [125].”
Deep Feature Interpolation for Image Content Changes (2016, background): “In concept, our work is similar to [41, 10, 42, 19, 17] that use video or photo collections to transfer the personality and character of one person’s face to a different person (a form of puppetry [39, 45, 20]).”
Production-level facial performance capture using deep convolutional neural networks (2016, methodology): “A lot of work has also been done for RGB-D sensors, such as Microsoft Kinect Fusion, e.g., [Bouaziz et al. 2013; Hsieh et al. 2015b; Li et al. 2013; Thies et al. 2015; Weise et al. 2011].”
InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image (2017, methodology): “A range of approaches use RGB-D input [e.g. 28, 44, 48], and while they achieve impressive face reconstruction results, they rely on depth data which is typically not available for in-the-wild images or videos.”
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis (2021, methodology): “Most approaches on this task rely on model-based facial performance capture [44, 45, 24, 23].”
FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces (2018, background): “[12] demonstrated the first real-time expression transfer for facial reenactment.”