HOPC: Histogram of Oriented Principal Components of 3D Pointclouds for Action Recognition

By Hossein Rahmani, Arif Mahmood, Du Q. Huynh, Ajmal Saeed Mian. 2014

194 citations recorded by the source corpus.

Existing techniques for 3D action recognition are sensitive to viewpoint variations because they extract features from depth images which change significantly with viewpoint. In contrast, we directly process the pointclouds and propose a new technique for action recognition which is more robust to noise, action speed and viewpoint variations. Our technique consists of a novel descriptor and keypoint detection algorithm. The proposed descriptor is extracted at a point by encoding the Histogram of Oriented Principal Components (HOPC) within an adaptive spatio-temporal support volume around that point. Based on this descriptor, we present a novel method to detect Spatio-Temporal Key-Points (STKPs) in 3D pointcloud sequences. Experimental results show that the proposed descriptor and STKP detector outperform state-of-the-art algorithms on three benchmark human activity datasets. We also introduce a new multiview public dataset and show the robustness of our proposed method to viewpoint variations.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In 3D Vision & Pose

Citations in the corpus

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (2016, neutral): “The IDs of training subjects in this evaluation are: 1, 2, 4, 5, 8, 9, 13, 14, 15, 16, 17, 18, 19, 25, 27, 28, 31, 34, 35, 38; remaining subjects are reserved for testing.”
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, neutral): “UWA3D Multiview [21] ∼900 30 10 1 Kinect v1 RGB+D+3DJoints 2014”
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, background): “As a result, depth sequences provide an exclusive modality of information about the 3D structure of the scene, which suits the problem of activity analysis [41, 47, 65, 73, 74, 79, 83].”
Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, background): “Among a large number of human representation approaches [24], [25], [26], [27], [28], [29], most of the existing 3D based methods can be broadly grouped into two categories: representations based on local features [30], [31] and skeletonbased representations [32] [33] [34].”
Multimodal Multipart Learning for Action Recognition in Depth Videos (2015, methodology): “Depth HOG [38] (as reported in [16]) 66.11% Actionlet Ensemble [2] (as reported in [16]) 82.22% HON4D [3] 96.67% Max Margin Time Warping [16] 97.22% HOPC [42] 98.33% Proposed MMMP 100.0% of each part by a magnitude based norm, and utilizing a soft part selection by a sparsity inducing norm.”
Histogram of Oriented Principal Components for Cross-View Action Recognition (2014, methodology): “Experimental results show that our method is able to achieve significantly better accuracy than current state-of-the-art methods for cross-view action recognition and is robust to action speed variations.”
PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding (2017, neutral): “MSR-Action3D [17] 20 567 567 1 D+Skeleton No 2010 RGBD-HuDaAct [21] 13 1189 1189 1 RGB+D No 2011 MSR-DailyActivity [43] 16 320 320 1 RGB+D+Skeleton No 2012 Act4 [8] 14 6844 6844 1 RGB+D No 2012 UTKinect-Action [53] 10 200 200 1 RGB+D+Skeleton No 2012 3D Action Pairs [23] 12 360 360 1 RGB+D+Skeleton ”
A novel hierarchical framework for human action recognition (2016, methodology): “The widely-used features 23 include sampled 3D points from silhouettes [6, 7], histograms of oriented 24 gradients [8], histogram of oriented 4D normals [9], histogram of oriented 25 principal components [10], etc.”
3D Action Recognition from Novel Viewpoints (2016, methodology): “The former is a deep CNN which represents different human body shapes and poses observed from numerous viewpoints in a viewinvariant high-level space.”
Learning a non-linear knowledge transfer model for cross-view action recognition (2015, background): “Figure 1: Existing cross-view action recognition techniques [11, 12, 19, 33, 40] connect source and target views with a set of linear transformations that are unable to capture the non-linear manifolds on which real actions lie.”
Large-scale Isolated Gesture Recognition using Convolutional Neural Networks (2016, unknown): “Since the first work of such a type [2] reported in 2010, many methods [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13] have been proposed based on specifical handcrafted feature descriptors extracted from depth/skeleton.”
A discussion on the validation tests employed to compare human action recognition methods using the MSR Action3D dataset (2014, methodology): “ Depth Motion Maps-based Local Binary Patterns, Chen et al. [26] 2015 98.1 92 94.6 94.9 91.94 HOPC: Histogram of Oriented Principal Components of 3D Pointclouds for Action Recognition, Rahmani et al. [27] 2014 - 91.64? Fusion of Skeletal and Silhouette-Based Features for Human Action Recognition wit”
Learning a Deep Model for Human Action Recognition from Novel Viewpoints (2016, unknown): “[3], [4], spatio-temporal interest points [5], [6], [7], [8], [9], [10], and motion trajectories based representation [11], [12], [13], [14].”
3D skeleton-based human action classification: A survey (2016, methodology): “Rahmani et al. [134] propose to use the histogram of oriented principal components (HOPC) to capture the local geometric characteristics around each point in the depth map by applying PCA to a volume of data around each point.”
Learning Action Recognition Model from Depth and Skeleton Videos (2017, methodology): “[20, 21] introduced to directly process the sequence of pointclouds corresponding to an action video.”
A Human Activity Recognition System Using Skeleton Data from RGBD Sensors (2016, background): “Other works proposed holistic descriptors: the HON4D descriptor [12], which is based on the orientations of normal surfaces in 4D, and HOPC descriptor [13], which is able to represent the geometric characteristics of a sequence of 3D points.”
Human Action Recognition From Various Data Modalities: A Review (2020, methodology): “Early methods [239], [253] focused on extracting hand-”