A Hierarchical Pose-Based Approach to Complex Action Understanding Using Dictionaries of Actionlets and Motion Poselets

By Ivan Lillo, Juan Carlos Niebles, Á. Soto. 2016

46 citations recorded by the source corpus.

In this paper, we introduce a new hierarchical model for human action recognition using body joint locations. Our model can categorize complex actions in videos, and perform spatio-temporal annotations of the atomic actions that compose the complex action being performed. That is, for each atomic action, the model generates temporal action annotations by estimating its starting and ending times, as well as, spatial annotations by inferring the human body parts that are involved in executing the action. Our model includes three key novel properties: (i) it can be trained with no spatial supervision, as it can automatically discover active body parts from temporal action annotations only, (ii) it jointly learns flexible representations for motion poselets and actionlets that encode the visual variability of body parts and atomic actions, (iii) a mechanism to discard idle or non-informative body parts which increases its robustness to common pose estimation errors. We evaluate the performance of our method using multiple action recognition benchmarks. Our model consistently outperforms baselines and state-of-the-art action recognition methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, 3D Vision & Pose, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

In Human-Computer Interaction

Citations in the corpus

Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection (2017, background): “Cheron et al. [4] used pose information to extract high-level features from appearance and optical flow.”
Global Context-Aware Attention LSTM Networks for 3D Action Recognition (2017, background): “Various feature extractors and classifier learning methods for 3D action recognition have been proposed in the past few years [28, 37, 31, 65, 54, 26, 34, 5, 47, 59, 38, 56, 32, 2].”
DAPs: Deep Action Proposals for Action Understanding (2016, background): “Spatio-Temporal Action Proposals: Recently, ideas from the area of object proposals have been extrapolated to action recognition in the video domain [6, 17, 11, 21, 24, 10, 39].”
RPAN: An End-to-End Recurrent Pose-Attention Network for Action Recognition in Videos (2017, background): “To improve the performance, several latent structures were proposed by learning meaningful hierarchical pose representations for action recognition [13, 22, 48].”