Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks

By Pichao Wang, Wanqing Li, Song Liu, Yuyao Zhang, Zhimin Gao, Philip O. Ogunbona. 2016

54 citations recorded by the source corpus.

This paper addresses the problem of continuous gesture recognition from sequences of depth maps using Convolutional Neural networks (ConvNets). The proposed method first segments individual gestures from a depth sequence based on quantity of movement (QOM). For each segmented gesture, an Improved Depth Motion Map (IDMM), which converts the depth sequence into one image, is constructed and fed to a ConvNet for recognition. The IDMM effectively encodes both spatial and temporal information and allows the fine-tuning with existing ConvNet models for classification without introducing millions of parameters to learn. The proposed method is evaluated on the Large-scale Continuous Gesture Recognition of the ChaLearn Looking at People (LAP) challenge 2016. It achieved the performance of 0.2655 (Mean Jaccard Index) and ranked 3rd place in this challenge.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, methodology): “Most of available datasets require subjects to perform actions in a visible and restricted view to avoid occlusion, and this results in limited view data collection and less 29 SBU Kinect Interaction [149] Depth+ Skeleton PRNN Side Information 89.20% [130] Skeleton Co-occurrence +LSTM None 90.41% [1”
Scene Flow to Action Map: A New Representation for RGB-D Based Action Recognition with Convolutional Neural Networks (2017, background): “The last category represents a video in one or multiple compact images and adopt available trained ConvNet architectures for finetuning [51, 52, 56, 1, 9, 53, 54].”
EgoGesture: A New Dataset and Benchmark for Egocentric Hand Gesture Recognition (2018, methodology): “We construct IDMMs as introduced in [38] by accumulating the absolute depth difference between current frame and the starting frame for each pre-segmented gesture samples.”
Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition (2016, background): “[32], which ranked third, proposed extracting improved depth motion maps from depth sequences and classifying them using 2D CNNs.”
Depth Pooling Based Large-Scale 3-D Action Recognition With Convolutional Neural Networks (2018, methodology): “IDMM + ConvNet [21] also adopted the action segmentation method and then extracted one improved depth motion map”
Large-scale Isolated Gesture Recognition using Convolutional Neural Networks (2016, methodology): “With the recent development of deep learning, a few methods have been developed based on Convolutional Neural Networks (ConvNets) [14], [15], [16], [17], [18] and Recurrent Neural Networks (RNNs) [19], [20], [21], [22].”