Multi-source Deep Learning for Human Pose Estimation

By Wanli Ouyang, Xiao Chu, Xiaogang Wang. 2014

265 citations recorded by the source corpus.

Visual appearance score, appearance mixture type and deformation are three important information sources for human pose estimation. This paper proposes to build a multi-source deep model in order to extract non-linear representation from these different aspects of information sources. With the deep model, the global, high-order human body articulation patterns in these information sources are extracted for pose estimation. The task for estimating body locations and the task for human detection are jointly learned using a unified deep model. The proposed approach can be viewed as a post-processing of pose estimation results and can flexibly integrate with existing methods by taking their information sources as input. By extracting the non-linear representation from multiple information sources, the deep model outperforms state-of-the-art by up to 8.6 percent on three public benchmark datasets.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields (2016, methodology): “To obtain reliable local observations of body parts, Convolu-tional Neural Networks (CNNs) have been widely used, and have signiﬁcantly boosted the accuracy on body pose estimation [23, 38, 24, 34, 35, 7, 36, 6].”
Convolutional Pose Machines (2016, background): “There has been a recent surge of interest in models that employ convolutional architectures for the task of articulated pose estimation [6, 7, 23, 24, 28, 38, 39].”
Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations (2014, neutral): “ples on the LSP and FLIC datasets. 6 Method Torso Head U.arms L.arms U.legs L.legs Mean Ours 92.7 87.8 69.2 55.4 82.9 77.0 75.0 Pishchulin et al. [16] 88.7 85.6 61.5 44.9 78.8 73.4 69.2 Ouyang et al. [14] 85.8 83.1 63.3 46.6 76.5 72.2 68.6 DeepPose* [23] - - 56 38 77 71 - Pishchulin et al. [15] 87.5”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, background): “Most traditional solutions to single-person pose estimation adopt the probabilistic graphical model or the pictorial structure model [76, 49], which is recently improved by exploiting deep learning for better modeling the unary and pair-wise energies [9, 63, 44] or imitating the iterative inference ”
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning (2020, background): “timodal when it includes a number of modalities. Multimodal representation learning frameworks can be divided into three types: (a) joint representation which aims to learn a shared semantic subspace [63,71,26,3,48,51,95]; (b) an encoder-decoder framework which aims to translate from one modality in”
Deep learning for visual understanding (2017, unknown): “[189] derived a multi-source deep model from a Deep Belief Net (DBN), which attempts to take advantage of three information sources of human articulation, i.”
Robust Optimization for Deep Regression (2015, methodology): “Recently, part-based models have been combined with deep learning for 2D human pose estimation [9, 30, 41], where deep part detectors serve as unary potential functions and also as image-based body prior for the computation of the pairwise potential functions.”
LCR-Net: Localization-Classification-Regression for Human Pose (2017, background): “Some recent approaches employ CNNs for 3D pose estimation in monocular images [5, 15, 23] or in videos [29, 37].”
Single-Pedestrian Detection Aided by Multi-pedestrian Detection (2013, unknown): “and deep models [48], [56], [57], [58], [59], [60], [70], [96], [97].”
DeepID-Net: Object Detection with Deformable Part Based Convolutional Neural Networks (2017, unknown): “Because of the power on learning feature representation, deep models have been widely used for object recognition, detection and other vision tasks [1], [6], [7], [19], [20], [21], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47],”
Combining local appearance and holistic view: Dual-Source Deep Neural Networks for human pose estimation (2015, methodology): “Following [17, 40], we visualize the patterns extracted by DS-CNN.”
Learning Deep Representation for Face Alignment with Auxiliary Attributes (2014, background): “DCN has been proven effective for many high-level computer vision tasks [16,42,28,24] and low-level vision problems [11].”
Human Pose Regression by Combining Indirect Part Detection and Contextual Information (2017, unknown): “iefel and Gehler [24] 83.5 73.7 55.9 36.2 73.7 70.5 66.9 65.8 Ramakrishnaet al. [35] 84.9 77.8 61.4 47.2 73.6 69.1 68.8 69.0 Pishchulinet al. [32] 87.5 77.6 61.4 47.6 79.0 75.2 68.4 71.0 Ouyanget al. [28] 86.5 78.2 61.7 49.3 76.9 70.0 67.6 70.0 Chen and Yuille [7] 91.5 84.7 70.3 63.2 82.7 78.1 72.0 ”
DeepID-Net: Deformable deep convolutional neural networks for object detection (2014, background): “Because of the power on learning feature representation, deep models have been widely used for object recognition, detection and other vision tasks [36, 54, 19, 37, 61, 17, 22, 14, 28, 30, 55, 56, 27, 23, 41, 43, 41, 42, 25, 26, 24, 58].”
A Survey of Deep Learning-Based Object Detection (2019, methodology): “Typical 2D human pose estimation methods [298]–[304] utilize deep CNN architectures.”
CRF-CNN: Modeling Structured Information in Human Pose Estimation (2016, methodology): “In contrast, our proposed full CRF-CNN model takes feature-output, output-output, and feature-feature relationships into consideration, which is novel in pose estimation.”