Learnable Triangulation of Human Pose

By Karim Iskakov, Egor Burkov, Victor S. Lempitsky, Yury Malkov. IEEE International Conference on Computer Vision, 2019

435 citations recorded by the source corpus.

We present two novel solutions for multi-view 3D human pose estimation based on new learnable triangulation methods that combine 3D information from multiple 2D views. The first (baseline) solution is a basic differentiable algebraic triangulation with an addition of confidence weights estimated from the input images. The second, more complex, solution is based on volumetric aggregation of 2D feature maps from the 2D backbone followed by refinement via 3D convolutions that produce final 3D joint heatmaps. Crucially, both of the approaches are end-to-end differentiable, which allows us to directly optimize the target metric. We demonstrate transferability of the solutions across datasets and considerably improve the multi-view state of the art on the Human3.6M dataset.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/a5feb080788bddb5ef4a25b342a9c3ab4aaa9aca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image (2020, neutral): “[16]* 20.”
Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation (2020, background): “…of L is n -norm, e.g., Mean-Square Error (MSE) loss and Smooth-(cid:96) 1 loss [22], which have been widely adopted in object detection [23], [24], pedestrian detection [25], [26], text detection [27]– [29], 3D detection [30], [31], pose estimation [32], [33], and instance segmentation [18], [20].”
MPDIoU: A Loss for Efficient and Accurate Bounding Box Regression (2023, background): “A typical form of L is `n-norm, for example, mean-square error (MSE) loss and Smooth-`1 loss [43], which have been widely adopted in object detection [44]; pedestrian detection [45, 46]; scene text spotting [34, 47]; 3D object detection [48, 49]; pose estimation [50, 51]; and instance segmentation [”
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (2020, methodology): “n SMPLify-X tting results and stateof-the-art 3D human pose estimation methods. \*" takes multi-view RGB images as inputs. methods MPJPE Moon et al. [42] 53.3 Sun et al. [58] 49.6 Iskakov et al. [22]* 20.8 SMPLify-X from GT 3D pose 13.1 13 Synthetic data from AMASS We leverage additional synthe”
Deep 3D human pose estimation: A review (2021, positive): “Typical meth-ods include fusing multi-view 2D heatmaps (Pavlakos et al., 2017b; Tome et al., 2018; Qiu et al., 2019), enforcing multiple view consistency (Rhodin et al., 2018a,b), triangulation (Kocabas et al., 2019; Iskakov et al., 2019), and utilizing the SMPL model (Liang and Lin, 2019).”
Atlas: End-to-End 3D Scene Reconstruction from Posed Images (2020, methodology): “Back projecting image features into a voxel volume and then refining them using a 3D CNN has also been used for human pose estimation [29, 59].”