Jointly Learning Heterogeneous Features for RGB-D Activity Recognition

By Jianfang Hu, Weishi Zheng, J. Lai, Jianguo Zhang. 2015

557 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (2016, methodology): “[15] proposed dynamic skeletons as Fourier temporal pyramids of spline-based interpolated skeleton points and their gradients, and HOG-based dynamic color and depth patterns to be used in a RGB+D joint-learning model for action classiﬁcation.”
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, methodology): “Among these approaches, the method in [26] uses the Fourier temporal pyramid [56] and the hand-crafted features for action classiﬁcation.”
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, methodology): “Multimodality analysis of RGB+D action videos is studied by [39, 34, 59, 20, 22, 87, 88, 31, 63, 23, 55, 34].”
RGB-D-based action recognition datasets: A survey (2016, methodology): “The SYSU 3D Human-Object Interaction dataset [38]( http://sist.sysu.edu.cn/ ~zhwshi/students/ jianfang/HomePage.htm ) was created by Sun Yat-sen University in 2015.”
Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, methodology): “Many other datasets captured using a Kinect v1 camera were also released to the public, including the MSR Daily Activity 3D [121], MSR Action Pairs [113], Online RGBD Action (ORGBD) [107], UTKinect-Action [124], Florence 3D-Action [118], CMU-MAD [104], UTD-MHAD [103], G3D/G3Di [105], [119], SPHERE [”
Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, neutral): “research domain [26], [27], and a series of hand-crafted features [28], [29], [30], [31], [32], [33], [34], [35], [36], [37] and deep learning-based approaches [18], [19], [38], [39], [40], [41], [42], [43], [44] have been proposed.”
Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points (2018, neutral): “0 Dynamic Skeletons [19] - 60.”
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks (2017, methodology): “We evaluate our proposed method on the NTU RGB+D [52], UT-Kinect [44], SYSU-3D [68], SBU-Kinect Interaction [69], and Berkeley MHAD [70] datasets.”
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates (2017, methodology): “2% Dynamic Skeletons [72] Geometric 75.”
Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks (2017, background): “, 3D skeletons representation in a Lie group [44], Fisher vector encoding of skeletal quads [10] and FTP dynamic [18].”
Interpretable 3D Human Action Analysis with Temporal Convolutional Networks (2017, positive): “Following the formulation of Equation 6, we observe that W2 acts as a gate that modulates how much information will be transformed and added on to X1.”
Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection (2017, positive): “, compares favorably Methods Cross Subject % Deep LSTM [32] 60.7% P-LSTM [32] 62.93% HOGˆ2 [25] 32.2% FTP DS [10] 60.23% ST-LSTM [21] 69.2% Ours (Pose) 67.8% Ours (RGB+OF+Pose - Baseline) 76.9% Ours (RGB+OF+Pose - Chained) 80.8 % to the existing methods.”
Learning and Refining of Privileged Information-Based RNNs for Action Recognition from Depth Sequences (2017, methodology): “CAD60 : We follow the same experimental setting as in [47, 20] by adopting the leave-one-person-out cross validation.”
View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data (2017, methodology): “We can see that our scheme signiﬁcantly outperforms the state-of-the-art approaches by about 6%, 4%, 1% in accuracy for the NTU, SBU, SYSU dataset respectively.”
A New Representation of Skeleton Sequences for 3D Action Recognition (2017, neutral): “Accuracy Cross Subject Cross View Lie Group [28] 50.1% 52.8% Skeletal Quads [5] 38.6% 41.4% Dynamic Skeletons [11] 60.2% 65.2% Hierarchical RNN [4] 59.1% 64.0% Deep RNN [24] 59.3% 64.1% Deep LSTM [24] 60.7% 67.3% Part-aware LSTM [24] 62.9% 70.3% ST-LSTM [16] 65.2% 76.1% ST-LSTM + Trust Gate [16] 69…”
Scene Flow to Action Map: A New Representation for RGB-D Based Action Recognition with Convolutional Neural Networks (2017, methodology): “These methods either extracted features from one modality: depth [49, 60, 29, 59, 24, 51, 52] or RGB [27, 33] or skeleton [43, 55, 56, 4, 34, 19], or fuse the features extracted separately from them at a later stage [28, 16, 10, 58].”
Unsupervised Learning of Long-Term Motion Dynamics for Videos (2017, methodology): “…and testing groups Methods Depth HOG [17] 32.24 Super Normal Vector [42] 31.82 HON4D [18] 30.56 Lie Group [32] 50.08 Skeletal Quads [3] 38.62 FTP Dynamic Skeletons [4] 60.23 HBRNN-L [2] 59.07 2 Layer P-LSTM [25] 62.93 Shufﬂe and Learn [15] 47.5 Our method (Unsupervised training) 66.2 Table 2.”
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition (2016, background): “Cross subject Cross view Lie Group [7] 50.1% 52.8% Skeletal Quads [6] 38.6% 41.4% Dynamic Skeletons [61] 60.2% 65.2% HBRNN [30] 59.1% 64.0% Part-aware LSTM [32] 62.9% 70.3% Deep RNN [32] 56.3% 64.1% Deep LSTM [32] 60.7% The results are shown in Table 1.”
An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data (2016, background): “However, in comparisons to the holistic based approaches (Simonyan and Zisserman 2014; Wu et al. 2015; Zhu et al. 2016) which use all the frames, it loses some information.”