PoseTrack: A Benchmark for Human Pose Estimation and Tracking

By Mykhaylo Andriluka, Anton Milan, Eldar Insafutdinov, Leonid Pishchulin, Juergen Gall, Bernt Schiele. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

530 citations recorded by the source corpus.

Existing systems for video-based pose estimation and tracking struggle to perform well on realistic videos with multiple people and often fail to output body-pose trajectories consistent over time. To address this shortcoming this paper introduces PoseTrack which is a new large-scale benchmark for video-based human pose estimation and articulated tracking. Our new benchmark encompasses three tasks focusing on i) single-frame multi-person pose estimation, ii) multi-person pose estimation in videos, and iii) multi-person articulated tracking. To establish the benchmark, we collect, annotate and release a new dataset that features videos with multiple people labeled with person tracks and articulated pose. A public centralized evaluation server is provided to allow the research community to evaluate on a held-out test set. Furthermore, we conduct an extensive experimental study on recent approaches to articulated pose tracking and provide analysis of the strengths and weaknesses of the state of the art. We envision that the proposed benchmark will stimulate productive research both by providing a large and representative training dataset as well as providing a platform to objectively evaluate and compare the proposed methods. The benchmark is freely accessible at https://posetrack.net/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/09b2e7af73689dbdba1547e19111a6ee06767906

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

VIBE: Video Inference for Human Body Pose and Shape Estimation (2019, methodology): “PennAction [69] and PoseTrack [3] are the only ground-truth 2D video datasets we use, while InstaVariety [30] and Kinetics-400 [31] are pseudo ground-truth datasets annotated using a 2D keypoint detector [12, 35].”
HOTA: A Higher Order Metric for Evaluating Multi-object Tracking (2020, methodology): “Many of these rank trackers using theMOTAmetric (PANDA [77], BDD100k [87], Waymo [73], ArgoVerse [14], PoseTrack [1], MOTS [74]), or a variation of MOTA (nuScenes [12], UA-DETRAC [78]).”
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time (2022, methodology): “PoseTrack In fact, multi-person pose tracking can be regarded as the combination of multi-person pose estimation and multi-object tracking.”
Whole-Body Human Pose Estimation in the Wild (2020, background): “There have been several body pose datasets [2,3,10,30,31,62].”
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (2019, background): “It is well-established [20, 32] that pre-training on large datasets followed by fine-tuning on target datasets boosts performance, especially when target datasets are small [3, 10, 26, 49].”
Pose Flow: Efficient Online Pose Tracking (2018, methodology): “PoseTrack Challenge Dataset is released in [ Andriluka and Iqbal, 2017 ] .”
Humans in 4D: Reconstructing and Tracking Humans with Transformers (2023, methodology): “Besides achieving state-of-the-art results for tracking on the PoseTrack dataset [1], we also apply HMR 2.0 on the downstream application of action recognition.”
The Computational Limits of Deep Learning (2020, background): “But the threshold should be adaptively selected based on the individual’s size [145].”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “Microsoft Common Objects in Context (COCO) Dataset [132] is the most widely used large-scale dataset.”
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (2020, background): “n articulation. Also, the 2D poses can be estimated accurately from in-the-wild images, since many well-performing methods [11, 43,56,64] are trained on large-scale in-the-wild 2D human pose datasets [2,34]. The second advantage is that Pose2Mesh avoids the representation issues of the pose paramete”
MotionBERT: A Unified Perspective on Learning Human Motion Representations (2022, methodology): “However, obtaining precise 2D pose annotations requires considerable effort [3], and acquiring ground-truth (GT) 3D joint locations is almost impossible.”
Deep 3D human pose estimation: A review (2021, unknown): “This dataset is collected based on the observations that the CNNs are often applied in isolated and separated datasets, such as MPII (Andriluka et al., 2014a), LSP (Johnson and Everingham, 2010), and are independent of 3D body estimation. To unite the people of multiple human datasets, the authors i”
MOTS: Multi-Object Tracking and Segmentation (2019, background): “PoseTrack [2] contains annotations of joint positions for multiple persons in videos.”