SSNet: Scale Selection Network for Online 3D Action Prediction

By Jun Liu, Amir Shahroudy, G. Wang, Ling-yu Duan, Alex ChiChung Kot. 2018

55 citations recorded by the source corpus.

In action prediction (early action recognition), the goal is to predict the class label of an ongoing action using its observed part so far. In this paper, we focus on online action prediction in streaming 3D skeleton sequences. A dilated convolutional network is introduced to model the motion dynamics in temporal dimension via a sliding window over the time axis. As there are significant temporal scale variations of the observed part of the ongoing action at different progress levels, we propose a novel window scale selection scheme to make our network focus on the performed part of the ongoing action and try to suppress the noise from the previous actions at each time step. Furthermore, an activation sharing scheme is proposed to deal with the overlapping computations among the adjacent steps, which allows our model to run more efficiently. The extensive experiments on two challenging datasets show the effectiveness of the proposed action prediction framework.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In 3D Vision & Pose

Citations in the corpus

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, methodology): “…[26], the Spatio-Temporal LSTM [48], the Internal Feature Fusion [94], the GCA-LSTM [30], the Multi-Task Learning Network [49], the FSNet [103], [108], the Skeleton Visualization [104], the Two-Stream Attention LSTM [105], the Multi-Task CNN with RotClips [106], and the Body Pose Evolution Map…”
Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, methodology): “In [72], the coordinates of the skeleton joints at each frame were simply concatenated to form a vector representing the current frame’s pose.”
Human Action Recognition From Various Data Modalities: A Review (2020, background): “, recognizing an action before it has been fully performed [418], [419], [420].”
Global Regularizer and Temporal-Aware Cross-Entropy for Skeleton-Based Early Action Recognition (2018, methodology): “[35] introduced a Scale Selection Network (SSNet) to effectively select different scales for online 3D early action recognition.”
Context Contrasted Feature and Gated Multi-scale Aggregation for Scene Segmentation (2018, background): “To ensure every information map be aware of global information, the outputs of RNN are concatnated , Hp = (h1p...h N p ) T , and refined with global information: Hp = Fg (Hp,Θg) + Hp (4) where Fg is a 1×1×N×N Conv and Θg is its parameters.”