Spatio-Temporal Laplacian Pyramid Coding for Action Recognition

By Ling Shao, Xiantong Zhen, Dacheng Tao, Xuelong Li. 2014

237 citations recorded by the source corpus.

We present a novel descriptor, called spatio-temporal Laplacian pyramid coding (STLPC), for holistic representation of human actions. In contrast to sparse representations based on detected local interest points, STLPC regards a video sequence as a whole with spatio-temporal features directly extracted from it, which prevents the loss of information in sparse representations. Through decomposing each sequence into a set of band-pass-filtered components, the proposed pyramid model localizes features residing at different scales, and therefore is able to effectively encode the motion information of actions. To make features further invariant and resistant to distortions as well as noise, a bank of 3-D Gabor filters is applied to each level of the Laplacian pyramid, followed by max pooling within filter bands and over spatio-temporal neighborhoods. Since the convolving and pooling are performed spatio-temporally, the coding model can capture structural and motion information simultaneously and provide an informative representation of actions. The proposed method achieves superb recognition rates on the KTH, the multiview IXMAS, the challenging UCF Sports, and the newly released HMDB51 datasets. It outperforms state of the art methods showing its great potential on action recognition.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mehrtash Harandi
2. Fatih Murat Porikli
3. Mengyang Yu
4. Ling Shao
5. Li Liu
6. Anan Liu
7. Ping-Ping Jia
8. Bhiksha Raj

In Biomedical Engineering

1. Li Liu
2. Fatih Murat Porikli
3. Anan Liu
4. Ling Shao
5. Mehrtash Harandi
6. Zhaoxuan Yang
7. Pietro Pala
8. Tong Hao

In Computer Vision and Pattern Recognition

1. Ling Shao
2. Luc Van Gool
3. Fatih Murat Porikli
4. Alan L. Yuille
5. Dacheng Tao
6. Mehrtash Harandi
7. Yi Yang
8. Ming-Hsuan Yang

Citations in the corpus

• Beyond Gaussian Pyramid: Multi-skip Feature Stacking for action recognition (2014, background): “However, without awareness of the fundamental differences between image and video processing, [30] was not very successful when compared to the state-of-the-art methods.”
• Going deeper into action recognition: A survey (2016, methodology): “A relevant idea is presented by Shao et al . where the Laplacian of 3D Gaussian filters is used to construct the action space Shao et al. (2014).”
• Multiple/Single-View Human Action Recognition via Part-Induced Multitask Structural Learning (2015, background): “datasets [11], [14], [26], [36], [42], [31] because such features can capture local salient characteristics of appearance and motion.”
• Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold (2014, methodology): “Besides methods in Euclidean spaces [25]–[27], some emerging and interesting techniques reformulate computer vision problems, like action recogni-”
• Structure-Preserving Binary Representations for RGB-D Action Recognition (2016, methodology): “Feature extraction from RGB video data has been well explored [17], [18], [19], [20].”