Mining Mid-Level Features for Action Recognition Based on Effective Skeleton Representation

By Pichao Wang, Wanqing Li, Philip O. Ogunbona, Zhimin Gao, Hanling Zhang. 2014

45 citations recorded by the source corpus.

Recently, mid-level features have shown promising performance in computer vision. Mid-level features learned by incorporating class-level information are potentially more discriminative than traditional low-level local features. In this paper, an effective method is proposed to extract mid-level features from Kinect skeletons for 3D human action recognition. Firstly, the orientations of limbs connected by two skeleton joints are computed and each orientation is encoded into one of the 27 states indicating the spatial relationship of the joints. Secondly, limbs are combined into parts and the limb's states are mapped into part states. Finally, frequent pattern mining is employed to mine the most frequent and relevant (discriminative, representative and non-redundant) states of parts in continuous several frames. These parts are referred to as Frequent Local Parts or FLPs. The FLPs allow us to build powerful bag-of-FLP-based action representation. This new representation yields state-of-the-art results on MSR DailyActivity3D and MSR ActionPairs3D.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, methodology): “[78] introduced a mining technique to find partbased mid-level patterns (frequent local parts) and aggregated the local representations as bag-of-FLPs to be classified by a SVM.”
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks (2017, neutral): “nition have been proposed [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41].”
Scene Flow to Action Map: A New Representation for RGB-D Based Action Recognition with Convolutional Neural Networks (2017, methodology): “These methods either extracted features from one modality: depth [49, 60, 29, 59, 24, 51, 52] or RGB [27, 33] or skeleton [43, 55, 56, 4, 34, 19], or fuse the features extracted separately from them at a later stage [28, 16, 10, 58].”
Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks (2016, methodology): “, Kinect) in 2010, many methods [2], [3], [4], [5], [6], [7], [8], [9] for action recognition have been proposed based on specific hand-crafted feature descriptors extracted from depth or skeleton data.”
Large-scale Isolated Gesture Recognition using Convolutional Neural Networks (2016, unknown): “Since the first work of such a type [2] reported in 2010, many methods [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13] have been proposed based on specifical handcrafted feature descriptors extracted from depth/skeleton.”
Action Recognition Based on Joint Trajectory Maps with Convolutional Neural Networks (2016, background): “Joint representation captures the correlation of the body joints by extracting spatial descriptor [30], [56]– [59], geometric descriptor [33], [34], [39]–[41], [60] or key poses [35], [61]–[63].”
Global Context-Aware Attention LSTM Networks for 3D Action Recognition (2017, background): “Various feature extractors and classifier learning methods for 3D action recognition have been proposed in the past few years [28, 37, 31, 65, 54, 26, 34, 5, 47, 59, 38, 56, 32, 2].”
Joint Distance Maps Based Action Recognition With Convolutional Neural Networks (2017, background): “In the past few years, many handcrafted skeleton features [4]–[14] have been proposed for action recognition.”