Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
By Raviteja Vemulapalli, Felipe Arrate, Ramalingam Chellappa. 2014
1,475 citations recorded by the source corpus.
Recently introduced cost-effective depth sensors coupled with the real-time skeleton estimation algorithm of Shotton et al. [16] have generated a renewed interest in skeleton-based human action recognition. Most of the existing skeleton-based approaches use either the joint locations or the joint angles to represent a human skeleton. In this paper, we propose a new skeletal representation that explicitly models the 3D geometric relationships between various body parts using rotations and translations in 3D space. Since 3D rigid body motions are members of the special Euclidean group SE(3), the proposed skeletal representation lies in the Lie group SE(3)×.. .×SE(3), which is a curved manifold. Using the proposed representation, human actions can be modeled as curves in this Lie group. Since classification of curves in this Lie group is not an easy task, we map the action curves from the Lie group to its Lie algebra, which is a vector space. We then perform classification using a combination of dynamic time warping, Fourier temporal pyramid representation and linear SVM. Experimental results on three action datasets show that the proposed representation performs better than many existing skeletal representations. The proposed approach also outperforms various state-of-the-art skeleton-based human action recognition approaches.
Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose, Human-Computer Interaction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biomedical Engineering
In Computer Vision and Pattern Recognition
In 3D Vision & Pose
- Jun Liu
- G. Wang
- Amir Shahroudy
- Hossein Rahmani
- Tae-Kyun Kim
- Alex ChiChung Kot
- Ling-yu Duan
- Mohammed Bennamoun
In Human-Computer Interaction
Citations in the corpus
- NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (2016, methodology): “In [37] different skeleton config-urations were represented as points on a Lie group.”
- NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, background): “In [54], different skeleton configurations were represented as points on a Lie group.”
- Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, methodology): “Recent single modality action recognition methods on depth signals can be divided into two major groups: depth map analysis methods [35, 41, 45, 79] and skeleton based methods [36, 65, 11, 73, 74].”
- Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, background): “[201] Lie Group Manifold Co Manif Hand X X X X Zhang and Parker [202] BIPOD Stat Body Hand X X X X X”
- Multimodal Multipart Learning for Action Recognition in Depth Videos (2015, neutral): “This way (8) will be expanded to:”
- Histogram of Oriented Principal Components for Cross-View Action Recognition (2014, methodology): “Keywords: Spatio-temporal keypoint, pointcloud, view invariance.”
- Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, methodology): “Most of the existing skeleton-based action recognition methods [30, 40, 45–48] receive the fully observed segmented videos as input (each sample contains one full action instance), and derive a class label.”
- Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points (2018, neutral): “Lie group [54] P 74.”
- When Kernel Methods Meet Feature Learning: Log-Covariance Network for Action Recognition From Skeletal Data (2017, methodology): “Last, but not least, we achieve a quite compact feature representation (the size of the FC is 512 at maximum), which is much much smaller with respect to other approaches, such as [18] or [35].”
- Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks (2017, background): “[46] represented each action as a curve in a Lie group, and then utlized a”
- Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates (2017, methodology): “5% Lie Group [9] (reported by [80]) Geometric 93.”
- Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks (2017, methodology): “Method Cross subject Cross view Lie Group [44] 50.”
- View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data (2017, methodology): “A general treatment employs a pre-processing step to transform the 3D joint coordinates from the camera coordinate system to a personcentric coordinate system by placing the body center at the origin, followed by rotating the skeleton such that the body plane is parallel to the (x, y)-plane, to make”
- PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding (2017, background): “Those local features includes rotations and translations to represent geometric relationships of body parts in a Lie group [39, 40], or the covariance matrix to learn the co-occurrence of skeleton points [14].”
- A New Representation of Skeleton Sequences for 3D Action Recognition (2017, methodology): “With the prevalence of highly-accurate and affordable devices, action recognition based on 3D skeleton sequence has been attracting increasing attention [32, 28, 4, 24, 36, 16, 14].”
- Scene Flow to Action Map: A New Representation for RGB-D Based Action Recognition with Convolutional Neural Networks (2017, methodology): “These methods either extracted features from one modality: depth [49, 60, 29, 59, 24, 51, 52] or RGB [27, 33] or skeleton [43, 55, 56, 4, 34, 19], or fuse the features extracted separately from them at a later stage [28, 16, 10, 58].”