Feature Boosting Network For 3D Pose Estimation

By Jun Liu, Henghui Ding, Amir Shahroudy, Ling-yu Duan, Xudong Jiang, G. Wang, Alex ChiChung Kot. 2019

129 citations recorded by the source corpus.

In this paper, a feature boosting network is proposed for estimating 3D hand pose and 3D body pose from a single RGB image. In this method, the features learned by the convolutional layers are boosted with a new long short-term dependence-aware (LSTD) module, which enables the intermediate convolutional feature maps to perceive the graphical long short-term dependency among different hand (or body) parts using the designed Graphical ConvLSTM. Learning a set of features that are reliable and discriminatively representative of the pose of a hand (or body) part is difficult due to the ambiguities, texture and illumination variation, and self-occlusion in the real application of 3D pose estimation. To improve the reliability of the features for representing each body part and enhance the LSTD module, we further introduce a context consistency gate (CCG) in this paper, with which the convolutional feature maps are modulated according to their consistency with the context representations. We evaluate the proposed method on challenging benchmark datasets for 3D hand pose estimation and 3D full body pose estimation. Experimental results show the effectiveness of our method that achieves state-of-the-art performance on both of the tasks.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, 3D Vision & Pose, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In 3D Vision & Pose

In Human-Computer Interaction

Citations in the corpus

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, background): “To have view-invariant representations of the actions, features can be extracted from the 3D body joint positions [51], [52] which are available for each frame.”
Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild (2020, background): “Numerous approaches have been proposed to regress a 3D pose from estimated 2D keypoints [62, 40, 32, 9].”
Adversarial Defense Via Local Flatness Regularization (2019, unknown): “Deep neural networks (DNNs) have been widely and successfully used in many computer vision area, such as pose estimation [1, 2, 3], object detection [4, 5, 6] and super-resolution [7, 8, 9].”
Semantic Segmentation With Context Encoding and Multi-Path Decoding (2020, background): “Recurrent Neural Networks (RNN) [23], [24], [46] are effective and efficient to learn such sequence relationship, thus all of the information maps are fed to RNN in sequence to learn the relationship of these information maps.”
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click (2020, background): “Recently, thanks to the great success of convolution neural networks (CNN) [36, 44, 55, 56, 63–70], CNN-based interactive segmentation methods have achieved exciting progress.”
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks (2019, methodology): “In this research, we follow the evaluation protocols in prior work [17, 21, 26, 28, 37, 39], in which 5 subjects (S1, S5, S6, S7, S8) are used for training and 2 subjects (S9 and S11) are adopted for testing.”