Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors

By Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qionghai Dai, Yebin Liu. Computer Vision and Pattern Recognition, 2021

409 citations recorded by the source corpus.

Human volumetric capture is a long-standing topic in computer vision and computer graphics. Although high-quality results can be achieved using sophisticated off-line systems, real-time human volumetric capture of complex scenarios, especially using light-weight setups, remains challenging. In this paper, we propose a human volumetric capture method that combines temporal volumetric fusion and deep implicit functions. To achieve high-quality and temporal-continuous reconstruction, we propose dynamic sliding fusion to fuse neighboring depth observations together with topology consistency. Moreover, for detailed and complete surface generation, we propose detailpreserving deep implicit functions for RGBD input which can not only preserve the geometric details on the depth inputs but also generate more plausible texturing results. Results and experiments show that our method outperforms existing methods in terms of view sparsity, generalization capacity, reconstruction quality, and run-time efficiency.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7e5fefa74f383ec2d77b72b639e3d6f30dea2970

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Gerard Pons-Moll
2. Yebin Liu
3. Zerong Zheng
4. Christian Theobalt
5. Bharat Lal Bhatnagar
6. Cristian Sminchisescu
7. Ilya A. Petrov
8. Xianghui Xie

In Computer Graphics and Computer-Aided Design

1. Gerard Pons-Moll
2. Bharat Lal Bhatnagar
3. Christian Theobalt
4. Cristian Sminchisescu
5. Yebin Liu
6. Zerong Zheng
7. Ilya A. Petrov
8. Xianghui Xie

In Computer Vision and Pattern Recognition

1. Gerard Pons-Moll
2. Christian Theobalt
3. Yebin Liu
4. Michael J. Black
5. Zerong Zheng
6. Bharat Lal Bhatnagar
7. Cristian Sminchisescu
8. Luc Van Gool

Citations in the corpus

• Tensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering (2022, methodology): “This method is later extended for telepresence and holographic communication [9,24,66].”
• BEHAVE: Dataset and Method for Tracking Human Object Interactions (2022, methodology): “Therefore, recent methods use RGBD [56,69,73,76,84] or volumetric data [9,10,19] for reliable human capture.”