InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image

By Hyeongwoo Kim, Michael Zollhöfer, Ayush Kumar Tewari, Justus Thies, Christian Richardt, Christian Theobalt. 2017

56 citations recorded by the source corpus.

We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image in a single shot. By estimating all these parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible. Previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created dataset. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. In addition, we propose an analysis-by-synthesis breeding approach which iteratively updates the synthetic training corpus based on the distribution of real-world images, and we demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Dermatology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

A Hybrid Model for Identity Obfuscation by Face Replacement (2018, neutral): “We zero-pad the image if its dimensions are smaller than the crop size, thereby obtaining the ﬁnal crop with the desired 256 × 256 size.”
3D Morphable Face Models—Past, Present, and Future (2019, unknown): “Currently, many approaches [Feng et al. 2018; Kim et al. 2018b; Klaudiny et al. 2017; McDonagh et al. 2016; Richardson et al. 2016; Sela et al. 2017; Yu et al. 2017] in the research community are trained on synthetic training data, since it is easy to acquire and comes by design with perfect…”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, methodology): “Our approach outperforms the deep learning techniques of [60] and [37].”