Anatomy-Aware 3D Human Pose Estimation With Bone-Based Pose Decomposition

By Tianlang Chen, Chengjie Fang, Xiaohui Shen, Zhili Chen, Jiebo Luo. IEEE transactions on circuits and systems for video technology (Print), 2021

329 citations recorded by the source corpus.

In this work, we propose a new solution to 3D human pose estimation in videos. Instead of directly regressing the 3D joint locations, we draw inspiration from the human skeleton anatomy and decompose the task into bone direction prediction and bone length prediction, from which the 3D joint locations can be completely derived. Our motivation is the fact that the bone lengths of a human skeleton remain consistent across time. This promotes us to develop effective techniques to utilize global information across all the frames in a video for high-accuracy bone length prediction. Moreover, for the bone direction prediction network, we propose a fully-convolutional propagating architecture with long skip connections. Essentially, it predicts the directions of different bones hierarchically without using any time-consuming memory units (e.g. LSTM). A novel joint shift loss is further introduced to bridge the training of the bone length and bone direction prediction networks. Finally, we employ an implicit attention mechanism to feed the 2D keypoint visibility scores into the model as extra guidance, which significantly mitigates the depth ambiguity in many challenging poses. Our full model outperforms the previous best results on Human3.6M and MPI-INF-3DHP datasets, where comprehensive evaluation validates the effectiveness of our model.

Subfields: Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism

View Paper: https://www.semanticscholar.org/paper/555a23a05777a0b03614b195c7c72f7c626a97e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Hong Liu
2. Junsong Yuan
3. Pichao Wang
4. Luc Van Gool
5. Jun Liu
6. Mengyuan Liu
7. G. Wang
8. Mubarak Shah

In Endocrinology, Diabetes and Metabolism

1. Hong Liu
2. Mengyuan Liu
3. Luc Van Gool
4. Nasser Kehtarnavaz
5. Pichao Wang

Citations in the corpus

• MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation (2021, methodology): “Following previous works [4, 24, 34, 38], we train a single model on five subjects (S1, S5, S6, S7, S8) and test it on two subjects (S9 and S11).”
• 3D Human Pose Estimation with Spatial and Temporal Transformers (2021, methodology): “Following previous works [32, 25, 5], we adopt the same experiment setting: all 15 actions are used for training and testing, the model is trained on five sections (S1, S5, S6, S7, S8) and tested on two subjects (S9 and S11).”
• Exploiting Temporal Contexts With Strided Transformer for 3D Human Pose Estimation (2021, methodology): “Following [20], [44], [45], we also report the mean per joint velocity error (MPJVE) corresponding to the MPJPE of the first derivative of the 3D pose sequences.”
• Deep Learning-based Human Pose Estimation: A Survey (2020, background): “[24] added a bone direction module and bone length module to ensure human anatomy temporal consistency across video frames, while Liu et al.”
• MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video (2022, methodology): “Although the proposed model can easily adapt to any length of input sequence, to be fair, we select some specific sequence lengths T for three datasets to compare our method with other methods which must have a certain 2D input length [4, 24, 32]: Human3.”