InverseFaceNet: Deep Monocular Inverse Face Rendering

By Hyeongwoo Kim, Michael Zollhöfer, Ayush Kumar Tewari, Justus Thies, Christian Richardt, Christian Theobalt. 2017

83 citations recorded by the source corpus.

We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image. By estimating all parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible in real time. Most previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. We further propose a self-supervised bootstrapping process in the network training loop, which iteratively updates the synthetic training corpus to better reflect the distribution of real-world imagery. We demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Dermatology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Timo Bolkart
2. Michael J. Black
3. Haiwen Feng
4. Radek Daněček
5. Yao Feng
6. Jiaolong Yang
7. Xin Tong
8. Yunde Jia

In Computer Vision and Pattern Recognition

1. Michael J. Black
2. Timo Bolkart
3. Jiaolong Yang
4. Dong Chen
5. Xiaogang Wang
6. Luc Van Gool
7. Haiwen Feng
8. Xiaoou Tang

In Dermatology

1. Michael J. Black
2. Timo Bolkart
3. Radek Daněček

Citations in the corpus

• Learning an animatable detailed 3D face model from in-the-wild images (2020, methodology): “…see [Morales et al. 2021; Zollhöfer et al. 2018]) enabling applications such as 3D avatar creation for VR/AR [Hu et al. 2017], video editing [Kim et al. 2018a; Thies et al. 2016], image synthesis [Ghosh et al. 2020; Tewari et al. 2020] face recognition [Blanz et al. 2002; Romdhani et al.…”
• EMOCA: Emotion Driven Monocular Face Capture and Animation (2022, background): “1 [44] Hyeongwoo Kim, Michael Zollhöfer, Ayush Tewari, Justus Thies, Christian Richardt, and Christian Theobalt.”
• Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set (2019, methodology): “[28] and an optimization-based approach of Garrido et al .”