A generative restricted Boltzmann machine based method for high-dimensional motion data modeling

By Siqi Nie, Ziheng Wang, Qiang Ji. 2015

53 citations recorded by the source corpus.

Extended RBM to model spatio-temporal patterns among high-dimensional motion data.Generative approach to perform classification using RBM, for both binary and multi-class classification.High classification accuracy in two computer vision applications: facial expression recognition and human action recognition. Many computer vision applications involve modeling complex spatio-temporal patterns in high-dimensional motion data. Recently, restricted Boltzmann machines (RBMs) have been widely used to capture and represent spatial patterns in a single image or temporal patterns in several time slices. To model global dynamics and local spatial interactions, we propose to theoretically extend the conventional RBMs by introducing another term in the energy function to explicitly model the local spatial interactions in the input data. A learning method is then proposed to perform efficient learning for the proposed model. We further introduce a new method for multi-class classification that can effectively estimate the infeasible partition functions of different RBMs such that RBM is treated as a generative model for classification purpose. The improved RBM model is evaluated on two computer vision applications: facial expression recognition and human action recognition. Experimental results on benchmark databases demonstrate the effectiveness of the proposed algorithm.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Pichao Wang
2. Wanqing Li
3. Philip O. Ogunbona
4. Zhimin Gao
5. Yonghong Hou
6. Chang-Fu Tang
7. Chuankun Li
8. Yuyao Zhang

In Signal Processing

1. Bo Li
2. Yuchao Dai
3. Wanqing Li
4. Philip O. Ogunbona
5. Zhimin Gao
6. Shuang Wang
7. Yonghong Hou
8. Mingyi He

Citations in the corpus

• Scene Flow to Action Map: A New Representation for RGB-D Based Action Recognition with Convolutional Neural Networks (2017, methodology): “These methods either extracted features from one modality: depth [49, 60, 29, 59, 24, 51, 52] or RGB [27, 33] or skeleton [43, 55, 56, 4, 34, 19], or fuse the features extracted separately from them at a later stage [28, 16, 10, 58].”
• Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep CNN (2017, methodology): “TABLE IV: Performance comparison on the G3D dataset [11] Method Accuracy ELC-KSVD [31] 82.37% Cov3DJ [32] 71.95% LRBM [38] 90.50% SOS [33] 95.45% JTM [21] 96.02% Proposed Method 93.9%”
• Spatially and Temporally Structured Global to Local Aggregation of Dynamic Depth Information for Action Recognition (2018, methodology): “Table 8 shows the performance of the proposed method with that reported in [17] and [50].”
• Action Recognition Based on Joint Trajectory Maps with Convolutional Neural Networks (2016, methodology): “Table VII compared the performance of the proposed method and those reported in [83].”
• Structured Images for RGB-D Action Recognition (2017, methodology): “For this dataset, the first 4 subjects are used for training, the fifth for validation and the remaining 5 subjects for testing, following the configuration in [21].”
• Skeleton Optical Spectra-Based Action Recognition Using Convolutional Neural Networks (2018, unknown): “for testing as configured in [26].”