A Real-Time and Hardware-Efficient Processor for Skeleton-Based Action Recognition With Lightweight Convolutional Neural Network

By Bingyi Zhang, Jun Han, Zhize Huang, Jianwei Yang, Xiaoyang Zeng. 2019

16 citations recorded by the source corpus.

Skeleton-based human action recognition (HAR) has been extensively studied these years because body skeleton has the simple but informative representation of human action, which greatly reduces the computation complexity compared with the image-based HAR. As a result, it is suitable for low power implementation in embedded platforms. In this brief, we present a systematic approach to developing a hardware-efficient and low-power processor for real-time skeleton-based HAR. First, a lightweight HAR algorithm only using the one-dimensional convolutional neural network (1D-CNN) is proposed. Second, the singular value decomposition is employed to compress the weights in the fully connected (FC) layers of the proposed convolutional neural network. Third, a hardware processor implementing the proposed algorithm is presented. Aimed at optimizing area and energy, this processor utilizes a flexible structure supporting different kernel sizes of the 1D-CNN and reuses hardware in both convolution layers and FC layers. The proposed processor is implemented under SMIC 65-nm CMOS technology and consumes a total area of 1.016 mm2. Experimental results show that the proposed processor can achieve state-of-the-art classification accuracy in NTU RGB+D dataset and SBU dataset while outperforming previous solutions in area and energy efficiency.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, positive): “them have been evaluated based on the preliminary version [47] of our dataset, or pre-trained on it for transfer learning for other tasks [43], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70], [71], [72], [73], [74], [75], [76], [77], [78], [79].”