P-CNN: Pose-Based CNN Features for Action Recognition

By Guilhem Chéron, Ivan Laptev, Cordelia Schmid. 2015

641 citations recorded by the source corpus.

This work targets human action recognition in video. While recent methods typically represent actions by statistics of local video features, here we argue for the importance of a representation derived from human pose. To this end we propose a new Pose-based Convolutional Neural Network descriptor (P-CNN) for action recognition. The descriptor aggregates motion and appearance information along tracks of human body parts. We investigate different schemes of temporal aggregation and experiment with P-CNN features obtained both for automatically estimated and manually annotated human poses. We evaluate our method on the recent and challenging JHMDB and MPII Cooking datasets. For both datasets our method shows consistent improvement over the state of the art.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

In Human-Computer Interaction

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, methodology): “Pose estimation and action recognition are usually handled as distinct problems [14] or the last is used as a prior for the first [57, 22].”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, background): “Possible solutions to handle viewpoint variation and occlusion include the use of multi-sensor systems (Chunhui et al., 2017; Ofli et al., 2013; Shahroudy et al., 2016; Wang et al., 2014). The multi-camera systems is able to generate multi-view data, but the drawback is the requirement of synchroniz”
Temporal Convolutional Networks for Action Segmentation and Detection (2016, background): “, [3, 39]), however, this task assumes the boundaries of each segment are known.”
Human Action Recognition From Various Data Modalities: A Review (2020, methodology): “[49] used the positions of human body joints to crop multiple human body parts from the RGB and optical flow images, which were passed through a two-stream network for feature extraction followed by a SVM classifier for HAR.”
Rank Pooling for Action Recognition (2015, background): “eural network architecture that learns both visual appearance and motion information in an unsupervised fashion on video 2 volumes. A human pose driven CNN feature extraction pipeline is presented in [5]. In [5], authors represent body regions of humans with motion-based and appearance-based CNN des”
Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation (2016, background): “[29] show that if temporal segmentation on MPII is known then CNN-based pose features achieve 71.4% mAP.”
RPAN: An End-to-End Recurrent Pose-Attention Network for Action Recognition in Videos (2017, methodology): “Different from previous approaches in pose-related action recognition [3, 5], the proposed RPAN can be trained in an end-to-end fashion with the total loss,”
AR-Net: Adaptive Frame Resolution for Efficient Action Recognition (2020, positive): “tion using shallow classication models such as SVM on top of local visual features extracted from a video [34,53]. In the context of deep neural networks, it is typically performed by either 2D-CNNs [31,46,10,17,21] or 3D-CNNs [49,7,25]. A straightforward but popular approach is the use of 2D-CNNs t”
Multi-region Two-Stream R-CNN for Action Detection (2016, methodology): “The results are significantly better than those of the IDT method [2] and the pose-based CNN method [21] which perform only classification.”