Deep Kinematic Pose Regression

By Xingyi Zhou, Xiao Sun, Shuang Liang, Yichen Wei. 2016

277 citations recorded by the source corpus.

Learning articulated object pose is inherently difficult because the pose is high dimensional but has many structural constraints. Most existing work do not model such constraints and does not guarantee the geometric validity of their pose estimation, therefore requiring a post-processing to recover the correct geometry if desired, which is cumbersome and sub-optimal. In this work, we propose to directly embed a kinematic object model into the deep neutral network learning for general articulated object pose estimation. The kinematic function is defined on the appropriately parameterized object motion variables. It is differentiable and can be used in the gradient descent based optimization in network training. The prior knowledge on the object geometric model is fully exploited and the structure is guaranteed to be valid. We show convincing experiment results on a toy example and the 3D human pose estimation problem. For the latter we achieve state-of-the-art result on Human3.6M dataset.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Human-Computer Interaction

Citations in the corpus

End-to-End Recovery of Human Shape and Pose (2017, methodology): “[52] directly regress joint rotations of a fixed kinematic tree.”
Semantic Graph Convolutional Networks for 3D Human Pose Regression (2019, methodology): “Most previous regression-based methods directly minimize the mean squared errors (MSE) of the predicted and ground truth joint positions [6, 34, 57, 76] or bone vectors [53].”
A Simple Yet Effective Baseline for 3d Human Pose Estimation (2017, methodology): “2d/3d positions Our first design choice is to use 2d and 3d points as inputs and outputs, in contrast to recent work that has used raw images [11,13,24,32,33,45,46,54,56] or 2d probability distributions [33, 56] as inputs, and 3d probabilities [33], 3d motion parameters [54] or basis pose coefficien”
Under review as a conference paper at ICLR 2020 3 L INEAR INEQUALITY CONSTRAINTS FOR DEEP LEARNING MODELS We consider a generic L layer neural network (2019, background): “Finally, several works have proposed handcrafted solutions for specific applications, such as skeleton prediction (Zhou et al., 2016) and prediction of rigid body motion (Byravan & Fox, 2017).”
Ordinal Depth Supervision for 3D Human Pose Estimation (2018, unknown): “[66] (ECCVW’16) 91.”
LCR-Net++: Multi-Person 2D and 3D Pose Detection in Natural Images (2018, unknown): “[65] (17 jts) - 107.”
On the Continuity of Rotation Representations in Neural Networks (2018, methodology): “Some works also used other 3D representations such as Euler angles and Lie algebra [19, 34], and penalized the joint position errors.”
HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation (2020, methodology): “[77] use the network to predict the rotation angle of each body joint, followed by an FK layer to generate the 3D joint coordinates.”
QuaterNet: A Quaternion-based Recurrent Model for Human Motion (2018, methodology): “This approach is inspired by [65] for hand tracking and [64] for human pose estimation in static images.”
StarMap for Category-Agnostic Keypoint and Viewpoint Estimation (2018, background): “Accurate semantic keypoint detection forms the basis for many visual understanding tasks, including human pose estimation [4,24,27,53], hand pose estimation [48,54], viewpoint estimation [26,37], feature matching [16], ﬁne-grained image classiﬁcation [49], and 3D reconstruction [38,41,11,10].”
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose (2016, result): “41mm for coordinate regression is comparable to recent reported results with a coordinate regression target output [19, 35, 22, 44].”
Deep 3D human pose estimation: A review (2021, unknown): “…input images to 3D body joint positions can be categorized into two classes: detection-based methods (Pavlakos et al., 2017a; Luvizon et al., 2018) and regression-based methods (Li and Chan, 2014; Zhou et al., 2016a; Sun et al., 2017; Tekin et al., 2017; Zhou et al., 2017; Luvizon et al., 2019).”
Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach (2017, methodology): “[33] propose to explicitly enforce the bone-length constraints in the prediction, using a generative forward-kinematic layer; Tekin et al.”
VNect (2017, unknown): “Request permissions from permissions@acm.org.”
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image (2017, methodology): “[22] incorporate model joint dependencies in the CNN via a max-margin formalism, others [48] impose kinematic constraints by embedding a dif-”
A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose (2021, background): “While feed-forward estimation of the 3D joint positions [29, 35, 37, 38, 46, 47, 55, 62, 67, 73, 76] or joint angles and bone lengths [56, 77] of the skeleton is highly accurate, such discriminative estimates are prone to misalignment when overlayed onto the input image due to the generalization gap”
Deep Learning-based Human Pose Estimation: A Survey (2020, unknown): “[149] embedded a kinematic model into a network as kinematic layers to enforce the orientation and rotation constrains.”
MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior (2017, unknown): “F”