One-shot-learning gesture recognition using HOG-HOF features

By Jakub Konecný, Michal Hagara. 2013

83 citations recorded by the source corpus.

The purpose of this paper is to describe one-shot-learning gesture recognition systems developed on the ChaLearn Gesture Dataset (ChaLearn). We use RGB and depth images and combine appearance (Histograms of Oriented Gradients) and motion descriptors (Histogram of Optical Flow) for parallel temporal segmentation and recognition. The Quadratic-Chi distance family is used to measure differences between histograms to capture cross-bin relationships. We also propose a new algorithm for trimming videos--to remove all the unimportant frames from videos. We present two methods that use a combination of HOG-HOF descriptors together with variants of a Dynamic Time Warping technique. Both methods outperform other published methods and help narrow the gap between human performance and algorithms on this task. The code is publicly available in the MLOSS repository.

Subfields: Computer Vision and Pattern Recognition, Developmental and Educational Psychology, Meta & Continual Learning, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Developmental and Educational Psychology

In Meta & Continual Learning

In Human-Computer Interaction

Citations in the corpus

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (2019, methodology): “Koneˇcn`y et al. [92] combined HOG and HOF descriptors together with the dynamic time warping technique for one-shot RGB+D-based action recognition.”
Explore Efficient Local Features from RGB-D Data for One-Shot Learning Gesture Recognition (2016, methodology): “For example, HOG and HOF features [19], [26], [27] are also widely used to extract features, and HMM [27] and DTW [20], [28] are commonly applied in temporal segmentation or gesture recognition from RGB-D data.”
Challenges in multimodal gesture recognition (2016, methodology): “The proposed method did not do as well as the previous methods (Konecny and Hagara, 2014; Wan et al., 2013; Jiang et al., 2015) on the CDG 2011 dataset, but nonetheless it contributes novel ideas for transfer learning, that can be useful when the number of training examples per class is limited.”
ChaLearn looking at people: A review of events and resources (2017, background): “Topic at Journal of Machine Learning Research [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30] 2016 TPAMI HuPBA: We organized a Special Issue on Multi-modal Human Pose Recovery and Behavior Analysis (HuPBA) at IEEE Transactions on Pattern Ana”
Large-scale gesture recognition with a fusion of RGB-D data based on the C3D model (2016, background): “In the early stage, the hand-crafted features like histogram of oriented gradients (HOG) and histogram of optical flow (HOF) are extracted [3] [4] [5], and the finite-state machine (FSM) [6], [7], hidden Markov model (HMM) [5] [8], and dynamic time warping (DTW) [9] are commonly applied in modeling ”
Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM (2017, methodology): “Konecnỳ and Hagara [27] used the dynamic time warping (DTW) method to recognize gestures based on the histograms of oriented gradients (HOG) and histograms of optical flow (HOF).”