Classifying and visualizing motion capture sequences using deep neural networks

By Kyunghyun Cho, X. Chen. 2013

84 citations recorded by the source corpus.

The gesture recognition using motion capture data and depth sensors has recently drawn more attention in vision recognition. Currently most systems only classify dataset with a couple of dozens different actions. Moreover, feature extraction from the data is often computational complex. In this paper, we propose a novel system to recognize the actions from skeleton data with simple, but effective, features using deep neural networks. Features are extracted for each frame based on the relative positions of joints (PO), temporal differences (TD), and normalized trajectories of motion (NT). Given these features a hybrid multi-layer perceptron is trained, which simultaneously classifies and reconstructs input data. We use deep autoencoder to visualize learnt features. The experiments show that deep neural networks can capture more discriminative information than, for instance, principal component analysis can. We test our system on a public database with 65 classes and more than 2,000 motion sequences. We obtain an accuracy above 95% which is, to our knowledge, the state of the art result for such a large dataset.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Liang Wang
2. Jun Liu
3. Wenjun Zeng
4. G. Wang
5. Cuiling Lan
6. Dacheng Tao
7. Luc Van Gool
8. Ming-Hsuan Yang

In Control and Systems Engineering

1. Liang Wang
2. Wenjun Zeng
3. Junliang Xing
4. C. Pal
5. Doyoung Kim
6. Yong Du
7. Sanghoon Lee
8. Yanghao Li

Citations in the corpus

• Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks (2016, methodology): “ains 2,337 skeleton sequences performed by 5 actors (184,046 frames after down-sampling). For fair comparison, we use the same protocol (65 classes, 10-fold cross validation) as used by Cho and Chen (Cho and Chen 2014). The pre-processing is the same as that done in the hierarchical RNN scheme (Du, ”
• Semi-supervised Learning with Encoder-Decoder Recurrent Neural Networks: Experiments with Motion Capture Sequences (2015, methodology): “The main difference with the one used by Cho & Chen (2013) is that we allow hips to rotate over an axis so that the actor may have a horizontal position (when lying down for example), while their method fixes the hips in a constant position and orientation.”
• Hierarchical recurrent neural network for skeleton based action recognition (2015, background): “Std. Cho and Chen, 2013 [4] 95.59 0.76 DURNN-T 94.63 1.16 DBRNN-T 94.79 1.11 DURNN-L 96.62 0.53 DBRNN-L 96.70 0.51 HURNN-L 96.70 0.41 HBRNN-L 96.92 0.50 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 1 3 5 7 9 1 1 1 3 1 5 1 7 1 9 2 1 2 3 2 5 2 7 2 9 3…”
• Human Action Recognition Based on MOCAP Information Using Convolution Neural Networks (2014, background): “The spatial location, temporal differences and normalized motion trajectories of human joints are considered to recognize human actions from skeletal information using deep neural network model [10].”
• Ensemble Deep Learning for Skeleton-Based Action Recognition Using Temporal Sliding LSTM Networks (2017, background): “[4] normalized the orientation of skeletons so that each and every skeleton could have its root at the origin.”
• WALKING WALKing walking: Action Recognition from Action Echoes (2017, methodology): “Following the same protocol in [Cho and Chen, 2014], we classify some samples of 130 actions into the same category, in which samples represent the same action (e.g., jogging starting from air and from floor).”