ARCH: Animatable Reconstruction of Clothed Humans

By Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, Tony Tung. Computer Vision and Pattern Recognition, 2020

384 citations recorded by the source corpus.

In this paper, we propose ARCH (Animatable Reconstruction of Clothed Humans), a novel end-to-end framework for accurate reconstruction of animation-ready 3D clothed humans from a monocular image. Existing approaches to digitize 3D humans struggle to handle pose variations and recover details. Also, they do not produce models that are animation ready. In contrast, ARCH is a learned pose-aware model that produces detailed 3D rigged full-body human avatars from a single unconstrained RGB image. A Semantic Space and a Semantic Deformation Field are created using a parametric 3D body estimator. They allow the transformation of 2D/3D clothed humans into a canonical space, reducing ambiguities in geometry caused by pose variations and occlusions in training data. Detailed surface geometry and appearance are learned using an implicit function representation with spatial local features. Furthermore, we propose additional per-pixel supervision on the 3D reconstruction using opacity-aware differentiable rendering. Our experiments indicate that ARCH increases the fidelity of the reconstructed humans. We obtain more than 50% lower reconstruction errors for standard metrics compared to state-of-the-art methods on public datasets. We also show numerous qualitative examples of animated, high-quality reconstructed avatars unseen in the literature so far.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0ff2c939d136df8988f845ae5cdfb725939a82ab

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction (2020, background): “Another two PIFu variations are PIFuHD [22] and ARCH [8].”
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction (2020, background): “The success of deep implicit representations in general object modeling has inspired research in 3D human reconstruction [8], [54], [55], [56], [57].”
A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose (2021, background): “A-NeRF bears close similarities with body models deﬁned implicitly in terms of level-sets [61] and density of a sum of Gaussians [21, 51, 52] that are used for reﬁning human pose, shape, and appearance via differentiable ray-tracing.”
ICON: Implicit Clothed humans Obtained from Normals (2021, methodology): “Given an RGB image of a clothed person, ARCH [19] and ARCH++ [18] reconstruct 3D human shape in a canonical space by warping query points from the canonical to the posed space, and projecting them onto the 2D image space.”
HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video (2022, methodology): “Other than free-viewpoint rendering, there is another related active research field that focus on human motion retargeting either in 2D [2, 7, 35, 43, 54, 67, 68] or 3D [19, 20, 25, 32, 49, 53, 69, 74].”
Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies (2021, methodology): “[18, 5] combine implicit function learning with the SMPL model to obtain detailed animatable human models.”