LCR-Net++: Multi-Person 2D and 3D Pose Detection in Natural Images

By Grégory Rogez, Philippe Weinzaepfel, Cordelia Schmid. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

309 citations recorded by the source corpus.

We propose an end-to-end architecture for joint 2D and 3D human pose estimation in natural images. Key to our approach is the generation and scoring of a number of pose proposals per image, which allows us to predict 2D and 3D poses of multiple people simultaneously. Hence, our approach does not require an approximate localization of the humans for initialization. Our Localization-Classification-Regression architecture, named LCR-Net, contains 3 main components: 1) the pose proposal generator that suggests candidate poses at different locations in the image; 2) a classifier that scores the different pose proposals; and 3) a regressor that refines pose proposals both in 2D and 3D. All three stages share the convolutional feature layers and are trained jointly. The final pose estimation is obtained by integrating over neighboring pose hypotheses, which is shown to improve over a standard non maximum suppression algorithm. Our method recovers full-body 2D and 3D poses, hallucinating plausible body parts when the persons are partially occluded or truncated by the image boundary. Our approach significantly outperforms the state of the art in 3D pose estimation on Human3.6M, a controlled environment. Moreover, it shows promising results on real images for both single and multi-person subsets of the MPII 2D pose benchmark and demonstrates satisfying 3D pose results even for multi-person images.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/97630b840eace24ede96e9e7a1e1089ff410c483

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation (2018, methodology): “Similar to model-based methods, learning approaches have benefited from the advent of robust 2D pose methods – by matching 2D detections to a 3D pose database [8, 66], by regressing pose from 2D joint distance matrices [35], by exploiting pose and geometric priors for lifting [69, 1, 51, 19, 32, 70,”
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop (2019, methodology): “Recent works have made signiﬁcant advances in the frontier of skeleton-based 3D human pose estimation from single images, with many approaches achieving impressive results [21, 23, 29, 33, 35, 45].”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “MuPoTS-3D 3DPCK ↑ Year Method All people Matched people 2019 [187] 70.”
A Survey of Deep Learning-Based Object Detection (2019, background): “[305] propose an end-to-end architecture for joint 2D and 3D human pose estimation in natural images which predicts 2D and 3D poses of multiple people simultaneously.”
Artificial Intelligence for the Metaverse: A Survey (2022, background): “[82] CNNs with dense layer connection and channel-attention connection.”
Human Pose Regression with Residual Log-likelihood Estimation (2021, background): “[50, 51] classify the human pose into a set of K anchor-poses and a regression module is proposed to refine the anchor to the final prediction.”
Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera (2018, background): “Very few approaches tackle multiple-person 3D pose estimation [31, 20].”
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image (2019, unknown): “Rogez et al .”
Deep 3D human pose estimation: A review (2021, unknown): “For multi-person estimation from a single monocular image, we introduce three two-stage methods (Rogez et al., 2019; Moon et al., 2019; Mehta et al., 2018) and a new single-stage method (Nie et al., 2019).”