Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network

By Sijin Li, Zhi-Qiang Liu, Antoni B. Chan. 2014

246 citations recorded by the source corpus.

We propose a heterogeneous multi-task learning framework for human pose estimation from monocular images using a deep convolutional neural network. In particular, we simultaneously learn a human pose regressor and sliding-window body-part and joint-point detectors in a deep network architecture. We show that including the detection tasks helps to regularize the network, directing it to converge to a good solution. We report competitive and state-of-art results on several datasets. We also empirically show that the learned neurons in the middle layer of our network are tuned to localized body parts.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

An Overview of Multi-task Learning (2018, background): “Moreover, similar to the multi-layer feedforward neural network, most deep-learning models [18–22] in this category treat the output of one hidden layer as the shared feature representation.”
A Survey on Multi-Task Learning (2017, background): “1, many deep MTL methods [17], [18], [19], [20], [21] assume that different tasks share the first several hidden layers and then have task-specific parameters in the subsequent layers.”
DenseBox: Unifying Landmark Localization with End to End Object Detection (2015, background): “Sijin et al. [21] simultaneously learn pose joint regressor and sliding window body part detector in a deep network architecture.”
Convolutional Models for Joint Object Categorization and Pose Estimation (2015, background): “Although fundamentally different to object pose estimation, some research has explored using CNNs to recognize human pose (Toshev & Szegedy, 2013; LI et al., 2014; Pﬁster et al., 2014).”
Rotating your face using multi-task deep neural network (2015, background): “Instead of sharing all weights, a DNN has the shared layers in the front part, followed by separated layers to perform different tasks [25, 13, 22].”
Towards Viewpoint Invariant 3D Human Pose Estimation (2016, background): “As a solution to this, researchers framed the problem as a multi-task learning problem where human joints must be first detected then precisely localized [41,17,42].”
3D Human Pose Estimation from Monocular Images with Deep Convolutional Neural Network (2014, methodology): “3 Joint point detection task Inspired by [25], we define a set of detection tasks for each joint i and each window l, where the goal is to predict the indicator variable,”
TransPose: Keypoint Localization via Transformer (2020, methodology): “[19, 32] perform gradient descent in the input space to find out what input patterns can maximize a given unit.”
A survey of deep neural network architectures and their applications (2017, unknown): “[98] proposed a heterogeneous multi-task W.”
Multitask learning and benchmarking with clinical time series data (2017, methodology): “gene expression data. Ngufor et al. [44] apply a similar framework to jointly solving thirteen related clinical tasks, including predicting mortality and length of stay. In computer vision, Li et al. [37] and Elhoseiny et al. [19] both showed that using a single model to perform joint detection (cla”
Synthesizing Training Images for Boosting Human 3D Pose Estimation (2016, methodology): “Gkioxari et al. [15] also explored a multi-task CNN, where an action classifier is combined with a 2D joints detector.”
SuperCNN: A Superpixelwise Convolutional Neural Network for Salient Object Detection (2015, background): “As shown in a recent work for pose estimation (Li et al. 2014), jointly training two networks, one for joint point regression and one for body part detection, is able to achieve superior performance compared with individually training each network.”
Learning Deep Representation for Face Alignment with Auxiliary Attributes (2014, background): “Index Terms—Face Alignment, face landmark detection, deep learning, convolutional network Ç”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “[119] proposed a heterogeneous multi-task framework that consists of two tasks: predicting joint coordinates from full images by a regressor and detecting body parts from image patches using a sliding window.”
Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach (2017, methodology): “Following the success of MTL in neural networks [20], [47], [48], we choose to use Deep Multi-task Learning (DMTL) for estimating multiple attributes from a single face image.”