GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians

By Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, Matthias Nießner. Computer Vision and Pattern Recognition, 2023

319 citations recorded by the source corpus.

We introduce GaussianAvatars11Project page: https://shenhanqian.github.io/gaussian-avatars, a new method to create photorealistic head avatars that are fully controllable in terms of expression, pose, and viewpoint. The core idea is a dynamic 3D representation based on 3D Gaussian splats that are rigged to a parametric morphable face model. This combination facilitates photorealistic rendering while allowing for precise animation control via the underlying parametric model, e.g., through expression transfer from a driving sequence or by manually changing the morphable model parameters. We parameterize each splat by a local coordinate frame of a triangle and optimize for explicit dis-placement offset to obtain a more accurate geometric representation. During avatar reconstruction, we jointly optimize for the morphable model parameters and Gaussian splat parameters in an end-to-end fashion. We demonstrate the animation capabilities of our photorealistic avatar in several challenging scenarios. For instance, we show reen-actments from a driving video, where our method outperforms existing works by a significant margin.

View Paper: https://www.semanticscholar.org/paper/6c0717e783f1f44c5ffa9aaad36076950ab8c444

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yinghao Xu
2. Ceyuan Yang
3. Yujun Shen
4. Bo Dai
5. Kerui Ren
6. Sida Peng
7. Lihan Jiang
8. Tao Lu

Citations in the corpus

• A Survey on 3D Gaussian Splatting (2024, unknown): “Current systems struggle to unify diverse physical behaviors ( e.g ., rigid, elastic, or soft-body dynamics) into cohesive simulations, handle complex multi-object interactions without manual intervention, and model scene-level interactions such as environmental feedback and dynamic lighting…”
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields (2024, unknown): “…3DGS has been quickly extended with respect to multiple domains, including anti-aliasing rendering [Yu et al. 2024], material modeling [Jiang et al. 2023; Shi et al. 2023], dynamic scene reconstruction [Yan et al. 2023], and animatable avatar creation [Qian et al. 2023; Zielonka et al. 2023].”
• Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians (2024, background): “With such strengths, it has significantly accelerated the process of replicating the real world into a digital counterpart [6]– [9], igniting the community’s imagination for scaling real-to-simulation environments [3], [10], [11].”
• GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation (2024, background): “This representation has been quickly adopted for various application, including image-or text-conditioned 3D [14,47,49] and 4D generation [52,74], avatar reconstruction [31,48,72,76,113], dynamic scene reconstruction [61,102,107,108], among others [11,21,97].”
• Recent advances in 3D Gaussian splatting (2024, unknown): “Thanks to the high efficiency of 3DGS, research works have flourished and pushed 3DGS into digital human creation.”