On Geometric Features for Skeleton-Based Action Recognition Using Multilayer LSTM Networks

By Songyang Zhang, Xiaoming Liu, Jun Xiao. 2017

312 citations recorded by the source corpus.

RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, background): “As a result, activity analysis with 3D skeleton data becomes a popular domain of research [9–17] thanks to its succinctness, high level representation, and robustness against variations in viewpoints, illumination, clothing textures, and background clutter [1, 18, 19].”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, methodology): “proaches [150, 174, 81], on several constrained simple datasets, such as MSR-Action3D, Berkeley MHAD and SBU Kinect Interaction.”
Coding Kendall's Shape Trajectories for 3D Action Recognition (2018, unknown): “0 JLd+RNN [44] – 95.”
SSNet: Scale Selection Network for Online 3D Action Prediction (2018, background): “As a result, activity analysis with 3D skeleton data becomes popular [19, 49, 62, 41, 81, 57, 43] due to its succinctness, high level representation, and robustness against variations in viewpoints, clothing textures, illumination, and background [11, 42, 23].”
Ensemble Deep Learning for Skeleton-Based Action Recognition Using Temporal Sliding LSTM Networks (2017, result): “97 %) compared with the previous best model [23] (95.”
Fusing Geometric Features for Skeleton-Based Action Recognition Using Multilayer LSTM Networks (2018, unknown): “It should be mentioned that this paper is an extension of our conference paper [22].”
Human Action Recognition From Various Data Modalities: A Review (2020, methodology): “Hence, various methods [155], [156], [157], [158] have applied and adapted RNNs and LSTMs to effectively model the temporal context information within the skeleton sequences for HAR.”
Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN (2018, methodology): “This performance is better than the state-of-the-art methods including those with attention models [40, 2] and other techniques [45, 30].”
Graph Distillation for Action Detection with Privileged Modalities (2017, background): “Specifically, with the availability of depth sensors and joint tracking algorithms, extensive research has been done on action classification and detection in RGB-D videos [39,47,48,61] as well as skeleton sequences [24,30,31,32,46,63].”
Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition (2018, background): “When increasing K 1 , K 2 , the local ﬁltering region will cover (Vemulapalli, Arrate, and Chellappa 2014) 52.76% 50.08% Dynamic Skeletons (Hu et al. 2015) 65.22% 60.23% HBRNN (Du, Wang, and Wang 2015) 63.97% 59.07% LieNet (Huang et al. 2016) 66.95% 61.37% Deep LSTM (Shahroudy et al. 2016) 67.29% 60”