Skeleton boxes: Solving skeleton based action detection with a single deep convolutional neural network

By Bo Li, Huahui Chen, Yucheng Chen, Yuchao Dai, Mingyi He. 2017

36 citations recorded by the source corpus.

Action recognition from well-segmented 3D skeleton video has been intensively studied. However, due to the difficulty in representing the 3D skeleton video and the lack of training data, action detection from streaming 3D skeleton video still lags far behind its recognition counterpart and image-based object detection. In this paper, we propose a novel approach for this problem, which leverages both effective skeleton video encoding and deep regression based object detection from images. Our framework consists of two parts: skeleton-based video image mapping, which encodes a skeleton video to a color image in a temporal preserving way, and an end-to-end trainable fast skeleton action detector (Skeleton Boxes) based on image detection. Experimental results on the latest and largest PKU-MMD benchmark dataset demonstrate that our method outperforms the state-of-the-art methods with a large margin. We believe our idea would inspire and benefit future research in this important area.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, background): “action detection approaches are developed for handling offline mode that conducts detection after observing the whole long sequence [22], [23], [68].”
Monocular human pose estimation: A survey of deep learning-based methods (2020, methodology): “Li et al. (2015b) designed an embedding sub-network learning latent pose structure information to guide the 3D joint coordinates mapping.”
SSNet: Scale Selection Network for Online 3D Action Prediction (2018, methodology): “Several methods attempted online detection [38], while most action detection approaches are developed for handling offline mode which conducts detection after observing the whole long sequence [48, 35, 56].”
A Real-Time and Hardware-Efficient Processor for Skeleton-Based Action Recognition With Lightweight Convolutional Neural Network (2019, methodology): “Skeleton Boxes [31] utilizes the VGG-16 with slight modification in its fully connected layer, and Clip+CNN+MTLN [30] makes use of VGG-19.”
Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation (2018, methodology): “3 Methods In all previous CNN-based methods [ Du et al. , 2016; Ke et al. , 2017; Li et al. , 2017b ] , joint coordinates are speci-ﬁed as channels.”