Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation

By Guanghan Ning, Zhi Zhang, Zhiquan He. 2017

180 citations recorded by the source corpus.

Human pose estimation using deep neural networks aims to map input images with large variations into multiple body keypoints, which must satisfy a set of geometric constraints and interdependence imposed by the human body model. This is a very challenging nonlinear manifold learning process in a very high dimensional feature space. We believe that the deep neural network, which is inherently an algebraic computation system, is not the most efficient way to capture highly sophisticated human knowledge, for example those highly coupled geometric characteristics and interdependence between keypoints in human poses. In this work, we propose to explore how external knowledge can be effectively represented and injected into the deep neural networks to guide its training process using learned projections that impose proper prior. Specifically, we use the stacked hourglass design and inception-resnet module to construct a fractal network to regress human pose images into heatmaps with no explicit graphical modeling. We encode external knowledge with visual features, which are able to characterize the constraints of human body models and evaluate the fitness of intermediate network output. We then inject these external features into the neural network using a projection matrix learned using an auxiliary cost function. The effectiveness of the proposed inception-resnet module and the benefit in guided learning with knowledge projection is evaluated on two widely used human pose estimation benchmarks. Our approach achieves state-of-the-art performance on both datasets.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, background): “The problem of human pose estimation has been intensively studied in the last years, from Pictorial Structures [2, 17, 37] to more recent CNN approaches [34, 25, 38, 20, 41, 54, 5, 51, 52, 36].”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, neutral): “[44] 98.”
Toward fast and accurate human pose estimation via soft-gated skip connections (2020, unknown): “, TMM’18[24] 98.”
Fast Human Pose Estimation (2018, background): “This boost L SP M P II is bigger than other state-of-the-art gains, e.g. +0.3% in 91.2% [21] vs 90.9% [19]; further +0.3% in 91.5% [23].”
Monocular human pose estimation: A survey of deep learning-based methods (2020, background): “…(3) Body structure constraint : (Tompson et al., 2014; Lifshitz et al., 2016; Yang et al., 2016; Gkioxari et al., 2016; Chu et al., 2016, 2017; Ning et al., 2018; Ke et al., 2018; Tang et al., 2018a; Tang and Wu, 2019) (4) Temporal constraint : (Jain et al., 2014; Pﬁster et al., 2015; Luo et…”
Associative Embedding: End-to-End Learning for Joint Detection and Grouping (2016, background): “Multiperson Pose Estimation Recent methods have made great progress improving human pose estimation in images in particular for single person pose estimation [50, 48, 52, 40, 8, 5, 41, 4, 14, 19, 34, 26, 7, 49, 44].”