I M Avatar: Implicit Morphable Head Avatars from Videos

By Yufeng Zheng, Victoria Fernández Abrevaya, Xu Chen, Marcel C. Buhler, Michael J. Black, Otmar Hilliges. Computer Vision and Pattern Recognition, 2021

300 citations recorded by the source corpus.

Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details. Neural volumetric representations approach photorealism but are hard to animate and do not generalize well to unseen expressions. To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose-related deformations via learned blendshapes and skinning fields. These attributes are pose-independent and can be used to morph the canonical geometry and texture fields given novel expression and pose parameters. We employ ray marching and iterative root-finding to locate the canonical surface intersection for each pixel. A key contribution is our novel analytical gradient formulation that enables end-to-end training of IMavatars from videos. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods. Code and data can be found at https://ait.ethz.ch/projects/2022/IMavatar/.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/792d86d09c2fe992d4ebfb5b80d6dd7ff76e1b3c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Advances in Neural Rendering (2021, background): “„I M Avatar” [ZAC ∗ 21] extends NeRFace based on skin-ning ﬁelds [CZB ∗ 21] which are used to deform the canonical NeRF volume given novel expression and pose parameters.”
GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (2023, background): “IMavatar [51] learns a 3D morphable head avatar with neural implicit functions, solving for a map from observed to canonical space via iterative root-finding.”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, unknown): “Methods such as [22] and [43] from section 3.1.2 approach the sparse view problem by using point clouds for depth supervision.”