Real-Time Online Action Detection Forests Using Spatio-Temporal Contexts

By Seungryul Baek, Kwang In Kim, Tae-Kyun Kim. 2016

24 citations recorded by the source corpus.

Online action detection (OAD) is challenging since 1) robust yet computationally expensive features cannot be straightforwardly used due to the real-time processing requirements and 2) the localization and classification of actions have to be performed even before they are fully observed. We propose a new random forest (RF)-based online action detection framework that addresses these challenges. Our algorithm uses computationally efficient skeletal joint features. High accuracy is achieved by using robust convolutional neural network (CNN)-based features which are extracted from the raw RGBD images, plus the temporal relationships between the current frame of interest, and the past and futures frames. While these high-quality features are not available in real-time testing scenario, we demonstrate that they can be effectively exploited in training RF classifiers: We use these spatio-temporal contexts to craft RF's new split functions improving RFs' leaf node statistics. Experiments with challenging MSRAction3D, G3D, and OAD datasets demonstrate that our algorithm significantly improves the accuracy over the state-of-the-art on-line action detection algorithms while achieving the real-time efficiency of existing skeleton-based RF classifiers.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, methodology): “Sliding window-based design [24], [25], [61], [69] and action proposals [60] have been adopted for action detection.”
Learning and Refining of Privileged Information-Based RNNs for Action Recognition from Depth Sequences (2017, background): “We attribute this faster learning ability to the mechanism of encoding PI [2], which allows us to distinguish the subtle depth difference across successive frames.”
Transition Forests: Learning Discriminative Temporal Transitions for Action Recognition and Detection (2016, background): “Recently, [16] introduced a more realistic dataset, baseline methods and shown state-of-the-art performance with a classification/regression RNN, later improved by [2] with the use of RGB-D spatio-temporal contexts and decision forests.”
SSNet: Scale Selection Network for Online 3D Action Prediction (2018, methodology): “The sliding window based design [77, 53, 2, 22] and action proposals [76] have been adopted for action detection.”