Extreme 3D Face Reconstruction: Looking Past Occlusions
By Anh Tuan Tran, Tal Hassner, Iacopo Masi, Eran Paz, Yuval Nirkin, Gérard G. Medioni. 2017
16 citations recorded by the source corpus.
Existing single view, 3D face reconstruction methods can produce beautifully detailed 3D results, but typically only for near frontal, unobstructed viewpoints. We describe a system designed to provide detailed 3D reconstructions of faces viewed under extreme conditions, out of plane rotations, and occlusions. Motivated by the concept of bump mapping, we propose a layered approach which decouples estimation of a global shape from its mid-level details (e.g., wrinkles). We estimate a coarse 3D face shape which acts as a foundation and then separately layer this foundation with details represented by a bump map. We show how a deep convolutional encoder-decoder can be used to estimate such bump maps. We further show how this approach naturally extends to generate plausible details for occluded facial regions. We test our approach and its components extensively, quantitatively demonstrating the invariance of our estimated facial details. We further provide numerous qualitative examples showing that our method produces detailed 3D face shapes in viewing conditions where existing state of the art often break down.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Yuval Nirkin
- Tal Hassner
- Anh Tuan Tran
- Yosi Keller
- Cuong V Nguyen
- Luc Van Gool
- Andrea Vedaldi
- Dacheng Tao
Citations in the corpus
- Generating 3D faces using Convolutional Mesh Autoencoders (2018, neutral): “These 3D face models have several applications including face recognition [40], generating and animating faces [28] and monocular 3D face reconstruction [44].”
- FSGAN: Subject Agnostic Face Swapping and Reenactment (2019, methodology): “Earlier methods relied on underlying 3D face representations [46] to transfer or con-trol facial appearances.”
- Transferability and Hardness of Supervised Classification Tasks (2019, background): “It is therefore reasonable that CE could be extended to non-discrete labeled tasks, such as, for faces, 3D reconstruction [58], pose estimation [10, 11] or segmentation [46].”