HuMoR: 3D Human Motion Model for Robust Pose Estimation

By Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, Leonidas J. Guibas. IEEE International Conference on Computer Vision, 2021

434 citations recorded by the source corpus.

We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape. Though substantial progress has been made in estimating 3D human motion and shape from dynamic observations, recovering plausible pose sequences in the presence of noise and occlusions remains a challenge. For this purpose, we propose an expressive generative model in the form of a conditional variational autoencoder, which learns a distribution of the change in pose at each step of a motion sequence. Furthermore, we introduce a flexible optimization-based approach that leverages HuMoR as a motion prior to robustly estimate plausible pose and shape from ambiguous observations. Through extensive evaluations, we demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset, and enables motion reconstruction from multiple input modalities including 3D keypoints and RGB(-D) videos. See the project page at geometry.stanford.edu/projects/humor.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/e137c4fcdd7be545f1001d5f590538387493f7f1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Humans in 4D: Reconstructing and Tracking Humans with Transformers (2023, background): “Although, there have been many approaches that estimate 3D human pose and shape relying on iterative optimization, e.g ., SM-PLify [7] and variants [22, 38, 56, 66, 72, 85], for this analysis we will focus on approaches that directly regress the body shape from a single image input.”
PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking (2023, background): “We also note that our human and animal motion profiles are limited by our base datasets, and this constraint could be lifted with the help of recent generative models [47, 64].”
GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping (2021, background): “Unlike HuMoR [53] where in each iteration only 1 future frame is generated, MNet’s generated motion improves when generating more number of future frames; note that the improvement saturates for 10 future frames, see Tab.”
EDGE: Editable Dance Generation From Music (2022, background): “joystick control [31], class-conditioning [15, 40], text-tomotion [41, 58], seed motions [8, 21, 43, 57]—is also an active area of study.”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, positive): “The NICE-SLAM also only used ∼ 1/4 of the FLOPs of iMAP, ∼ 1/3 the tracking time and ∼ 1/2 the mapping time.”
Executing your Commands via Motion Diffusion in Latent Space (2022, background): “Human Motion Synthesis allows rich inputs of multimodal data, such as text [16, 47, 68, 18, 2, 28], action category [46, 19], incomplete pose sequences [12, 20, 68], control signals [65, 66, 45], musics [34, 33, 32] and image(s) [55, 9], here, we focus on some typical tasks.”