Multi-view Body Part Recognition with Random Forests

By Vahid Kazemi, Magnus Burenius, Hossein Azizpour, Josephine Sullivan. 2013

96 citations recorded by the source corpus.

Many computer vision tasks such as object detection, pose estimation,and alignment are directly related to the estimation of correspondences overinstances of an object class. Other tasks such as image classification andverification if not completely solved can largely benefit from correspondenceestimation. This thesis presents practical approaches for tackling the corre-spondence estimation problem with an emphasis on deformable objects.Different methods presented in this thesis greatly vary in details but theyall use a combination of generative and discriminative modeling to estimatethe correspondences from input images in an efficient manner. While themethods described in this work are generic and can be applied to any object,two classes of objects of high importance namely human body and faces arethe subjects of our experimentations.When dealing with human body, we are mostly interested in estimating asparse set of landmarks – specifically we are interested in locating the bodyjoints. We use pictorial structures to model the articulation of the body partsgeneratively and learn efficient discriminative models to localize the parts inthe image. This is a common approach explored by many previous works. Wefurther extend this hybrid approach by introducing higher order terms to dealwith the double-counting problem and provide an algorithm for solving theresulting non-convex problem efficiently. In another work we explore the areaof multi-view pose estimation where we have multiple calibrated cameras andwe are interested in determining the pose of a person in 3D by aggregating2D information. This is done efficiently by discretizing the 3D search spaceand use the 3D pictorial structures model to perform the inference.In contrast to the human body, faces have a much more rigid structureand it is relatively easy to detect the major parts of the face such as eyes,nose and mouth, but performing dense correspondence estimation on facesunder various poses and lighting conditions is still challenging. In a first workwe deal with this variation by partitioning the face into multiple parts andlearning separate regressors for each part. In another work we take a fullydiscriminative approach and learn a global regressor from image to landmarksbut to deal with insufficiency of training data we augment it by a large numberof synthetic images. While we have shown great performance on the standardface datasets for performing correspondence estimation, in many scenariosthe RGB signal gets distorted as a result of poor lighting conditions andbecomes almost unusable. This problem is addressed in another work wherewe explore use of depth signal for dense correspondence estimation. Hereagain a hybrid generative/discriminative approach is used to perform accuratecorrespondence estimation in real-time.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, background): “[112] contains images of professional football players during real matches, which are obtained using color sensors from 3 views.”
A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System (2018, methodology): “When seen as only a silhouette, it is not possible to infer if the mannequin is facing towards or away from the camera recent pose-recognition algorithms [94 – 97], or by coupling generative methods with discriminative methods [98].”
Robust Optimization for Deep Regression (2015, methodology): “Volleyball: Similar to the Football dataset [20], our results on the Volleyball dataset are already quite competitive using one ConvNet (Table 1d), with a PCP score of 81.”
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose (2016, background): “6M [14], HumanEva-I [30] and KTH Football II [15].”
Ordinal Depth Supervision for 3D Human Pose Estimation (2018, background): “In contrast, the complexity of collecting images with corresponding 3D ground truth has constrained 3D human pose datasets in small scale [19] or strictly in studio settings [42, 16].”
Soccer on Your Tabletop (2018, unknown): “8, we illustrate the estimated meshes on the KTH multiview soccer dataset [23], with a player captured from three different, synced cameras.”
Learnable Triangulation of Human Pose (2019, background): “6M validation scenes and scenes from KTH Multiview Football Dataset [8] (see our project page for the qualitative results).”
MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior (2017, unknown): “6M [37], Human Eva I [65], KTH Football II [66], and MPII [62].”
3D Pictorial Structures for Multiple Human Pose Estimation (2014, background): “[17] introduces better 2D part detectors based on learning with randomized forest classifiers, but still relies on the optimization proposed in 3D pictorial structures work [8].”
3D Human pose estimation: A review of the literature and analysis of covariates (2016, background): “6M dataset [143, 152] or the KTH Multiview Football Dataset II [144]).”