Human activity recognition from 3D data: A review

By Jake K. Aggarwal, Lu Xia. 2014

562 citations recorded by the source corpus.

Abstract Human activity recognition has been an important area of computer vision research since the 1980s. Various approaches have been proposed with a great portion of them addressing this issue via conventional cameras. The past decade has witnessed a rapid development of 3D data acquisition techniques. This paper summarizes the major techniques in human activity recognition from 3D data with a focus on techniques that use depth data. Broad categories of algorithms are identified based upon the use of different features. The pros and cons of the algorithms in each category are analyzed and the possible direction of future research is indicated.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (2016, background): “3D object recognition, 3D scene understanding, and 3D action recognition [1].”
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, background): “This empowers the computer vision solutions to move important steps towards 3D vision, such as 3D object recognition [2], 3D scene understanding [3], and 3D activity analysis [4], etc .”
A self-organizing neural network architecture for learning human-object interactions (2017, background): “Applications of this type of technology have led to the successful classiﬁcation of full-body actions and recognition of hand gestures [26].”
RGB-D-based action recognition datasets: A survey (2016, background): “However, available RGB-D-based datasets have insofar only been briefly summarised or enumerated without comprehensive coverage and in-depth analysis in the survey papers, such as [9,10], that mainly focus on the development of RGB-D-based action recognition algorithms.”
Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, background): “For example, Xia et al. [133] partitioned the 3D space into a number of bins using a modified spherical coordinate system and counted the number of joints falling in each bin to form a 1D histogram, which is called the Histogram of 3D Joint Positions (HOJ3D).”
Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, neutral): “research domain [26], [27], and a series of hand-crafted features [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and deep learning-based approaches [18], [19], [38], [39], [40], [41], [42], [43], [44] have been proposed.”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, background): “Several survey papers have summarized the research on human motion recognition using RGB-D data [14, 166, 3, 16, 172, 28, 100, 46].”
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks (2017, methodology): “With the development of depth sensors, such as Kinect and RealSense, 3D skeleton based human action recognition has received much attention, and a lot of advanced methods have been proposed during the past few years [4], [5], [6], [7].”
Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks (2017, background): “Previous skeleton based action recognition methods mainly focus on handcrafted features [1].”
Interpretable 3D Human Action Analysis with Temporal Convolutional Networks (2017, background): “Applications of human activity recognition ranges from video surveillance, humancomputer interaction, robotics and skill evaluation [2, 35].”
View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data (2017, background): “tion for action recognition has recently attracted a lot of research attention because of its high level representation and robustness to variations of viewpoints, appearances, and surrounding distractions [2, 10, 28, 47].”
PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding (2017, background): “For a more extensive conclusion of activity analysis we refer to corresponding survey papers [5, 56, 1, 7].”
Human Action Recognition From Various Data Modalities: A Review (2020, methodology): “Some survey papers reviewed methods that use the skeleton modality for HAR [35], [36], [37], [38].”
A Human Activity Recognition System Using Skeleton Data from RGBD Sensors (2016, background): “In [6], the authors found that many features can be computed from skeleton joints.”
Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning (2018, background): “Over the years, skeleton-based human action recognition has attracted more and more attention [2, 4, 26].”
Skeletal Quads: Human Action Recognition Using Joint Quadruples (2014, background): “We consider a local coordinate system, such that x1 becomes the origin and x2 is mapped onto [1, 1, 1] .”
3D skeleton-based human action classification: A survey (2016, result): “In contrast to very recent surveys such as [66], [67] which focus on the use of depths cameras in several application domains or present a review of the literature on activity recognition in depth maps, in this survey”
Ensemble Deep Learning for Skeleton-Based Action Recognition Using Temporal Sliding LSTM Networks (2017, background): “With a rapid development of 3D data acquisition over the past few decades, lots of researches on human activity recognition from 3D data can have been actively performed [2].”