DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

By Leonid Pishchulin, Eldar Insafutdinov, Siyu Tang, Bjoern Andres, Mykhaylo Andriluka, Peter Gehler, Bernt Schiele. 2015

1,029 citations recorded by the source corpus.

This paper considers the task of articulated human pose estimation of multiple people in real world images. We propose an approach that jointly solves the tasks of detection and pose estimation: it infers the number of persons in a scene, identifies occluded body parts, and disambiguates body parts between people in close proximity of each other. This joint formulation is in contrast to previous strategies, that address the problem by first detecting people and subsequently estimating their body pose. We propose a partitioning and labeling formulation of a set of body-part hypotheses generated with CNN-based part detectors. Our formulation, an instance of an integer linear program, implicitly performs non-maximum suppression on the set of part candidates and groups them to form configurations of body parts respecting geometric and appearance constraints. Experiments on four different datasets demonstrate state-of-the-art results for both single person and multi person pose estimation.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Towards Accurate Multi-person Pose Estimation in the Wild (2017, methodology): “For this we use a Resnet [21] applied in a fully convolutional fashion to predict activation heatmaps and offsets for each keypoint, similar to the works of Pishchulin et al. [33] and Insafutdinov et al. [24], followed by combining their predictions using a novel form of heatmap-offset aggregation.”
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields (2016, methodology): “Table 1 compares mAP performance between our method and other approaches on the same subset of 288 testing images as in [27], and the entire MPI testing set.”
Stacked Hourglass Networks for Human Pose Estimation (2016, background): “[28–30] Also, there is the more challenging task of simultaneous annotation of multiple people [17,31].”
2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, background): “The problem of human pose estimation has been intensively studied in the last years, from Pictorial Structures [2, 17, 37] to more recent CNN approaches [34, 25, 38, 20, 41, 54, 5, 51, 52, 36].”
Deep High-Resolution Representation Learning for Visual Recognition (2019, background): “Other works include: light upsample process [7], [24], [92], [152], possibly with dilated convolutions used in the back-bone [63], [89], [113]; light downsample and heavy up-sample processes [141], recombinator networks [55]; improving skip connections with more or complicated convolutional units…”
DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model (2016, methodology): “…are three-fold, leading to a novel multi-person pose estimation approach that is deeper, stronger, and faster compared to the state of the art [10]: – “deeper”: we propose strong body part detectors based on recent advances in deep learning [12] that – taken alone – already allow to obtain…”
Recurrent Human Pose Estimation (2016, neutral): “. [38] 95.8 90.3 80.5 74.3 77.6 69.7 62.8 79.6 51.8 Carreira et al. [9] 95.7 91.7 81.7 72.4 82.8 73.2 66.4 81.3 49.1 Tompson et al. [37] 96.1 91.9 83.9 77.8 80.9 72.3 64.8 82.0 54.9 Pishchulin et al. [29] 94.1 90.2 83.4 77.3 82.6 75.7 68.6 82.4 56.5 Insafutdinov et al. [18] 96.8 95.2 89.3 84.4 88.4 ”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, methodology): “In [26, 50, 34], dilated convolutions are adopted in the last two stages in the ResNet or VGGNet to eliminate the spatial resolution loss, which is followed by a light lowto-high process to further increase the resolution, avoiding expensive computation cost for only using dilated convolutions [11, ”
Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning (2018, background): “Due to its compelling utility for HCI, motion capture, and security applications, a large body of work has since developed in this joint detection domain [53, 52, 39, 37, 62, 38, 20, 14].”
Associative Embedding: End-to-End Learning for Joint Detection and Grouping (2016, methodology): “Evaluation is performed on MPII Multi-Person, a set of 1758 groups of multiple people taken from the test set as outlined in [35].”
Joint Graph Decomposition and Node Labeling by Local Search (2016, background): “Articulated human body pose estimation [20, 9] is the task of deciding, for every point in an image, whether this point depicts a part of the human body, and of deciding, for every pair of points that depict body parts, if they belong to the same body.”
Deep 3D human pose estimation: A review (2021, positive): “As a well-known method, the SMPLify (Bogo et al., 2016) first estimates 2D keypoints using DeepCut (Pishchulin et al., 2016) and then fits the SMPL model to these keypoints.”
Monocular 3D Human Pose Estimation by Predicting Depth on Joints (2017, methodology): “For in-the-wild 2D pose dataset, we use the MPII-LSP-extended dataset [21] which is a combination of the extend LSP [12] and the MPII dataset [1].”
Benchmarking and Error Diagnosis in Multi-instance Pose Estimation (2017, methodology): “However, none of the currently adopted benchmarks for Multi-Person pose estimation [14, 27, 31] carry out an extensive error and performance analysis speciﬁc to this framework, and mostly rely on the metrics from the Single-Person case.”