Histogram of Oriented Principal Components for Cross-View Action Recognition

By Hossein Rahmani, Arif Mahmood, Du Q. Huynh, Ajmal Saeed Mian. 2014

194 citations recorded by the source corpus.

Existing techniques for 3D action recognition are sensitive to viewpoint variations because they extract features from depth images which are viewpoint dependent. In contrast, we directly process point clouds for cross-view action recognition from unknown and unseen views. We propose the histogram of oriented principal components (HOPC) descriptor that is robust to noise, viewpoint, scale and action speed variations. At a 3D point, HOPC is computed by projecting the three scaled eigenvectors of the pointcloud within its local spatio-temporal support volume onto the vertices of a regular dodecahedron. HOPC is also used for the detection of spatiotemporal keypoints (STK) in 3D pointcloud sequences so that view-invariant STK descriptors (or Local HOPC descriptors) at these key locations only are used for action recognition. We also propose a global descriptor computed from the normalized spatio-temporal distribution of STKs in 4-D, which we refer to as STK-D. We have evaluated the performance of our proposed descriptors against nine existing techniques on two cross-view and three single-view human action recognition datasets. The experimental results show that our techniques provide significant improvement over state-of-the-art methods.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (2016, background): “The work of [26] proposed histograms of oriented principle components of depth cloud points, in order to extract robust features against viewpoint variations.”
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, background): “The work of [24] introduced histograms of oriented principle components of depth cloud points to extract robust features against viewpoint variations.”
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, background): “Recent single modality action recognition methods on depth signals can be divided into two major groups: depth map analysis methods [35, 41, 45, 79] and skeleton based methods [36, 65, 11, 73, 74].”
PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding (2017, neutral): “MSR-Action3D [17] 20 567 567 1 D+Skeleton No 2010 RGBD-HuDaAct [21] 13 1189 1189 1 RGB+D No 2011 MSR-DailyActivity [43] 16 320 320 1 RGB+D+Skeleton No 2012 Act4 [8] 14 6844 6844 1 RGB+D No 2012 UTKinect-Action [53] 10 200 200 1 RGB+D+Skeleton No 2012 3D Action Pairs [23] 12 360 360 1 RGB+D+Skeleton ”
Human Action Recognition From Various Data Modalities: A Review (2020, methodology): “Early methods [239], [253] focused on extracting hand-”
Learning features combination for human action recognition from skeleton sequences (2017, methodology): “Some methods use solely depth maps for action recognition [9, 11, 14, 16, 28].”
View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition (2018, methodology): “We follow the standard protocol proposed by [35] to evaluate the performance.”
Dual Contrastive Prediction for Incomplete Multi-View Representation Learning (2022, methodology): “UWA (UWA3D Multi-view Activity) [57]: The dataset is collected by Kinect sensors with RGB and depth features.”
First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations (2017, methodology): “s. These methods usually focus on how to extract discriminative features from the depth images using local geometric descriptors [40,43,76] sensitive to viewpoint changes and viewinvariant approaches [46,47]. However, the recent trend is to take advantage of the depth channel to obtain robust body p”
MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain (2022, background): “These datasets are often composed of RGB images or videos as well as other modalities such as depth [56, 95, 88, 49, 9, 75, 44, 61], especially after the release of the Microsoft Kinect [102].”
Modality Distillation with Multiple Stream Networks for Action Recognition (2018, methodology): “– we propose a new multimodal stream network architecture able to exploit multiple data modalities at training while using only one at test time; – we introduce a new paradigm to learn an hallucination network within a novel two-stream model; – in this context, we have implemented an inter-stream co”
3D Action Recognition from Novel Viewpoints (2016, methodology): “The former is a deep CNN which represents different human body shapes and poses observed from numerous viewpoints in a viewinvariant high-level space.”
Learning a Deep Model for Human Action Recognition from Novel Viewpoints (2016, methodology): “Figures 14 and 15 show the recognition accuracy on IXMAS and UWA3DII datasets respectively for all possible source|target view pairs.”