Tensor Representations via Kernel Linearization for Action Recognition from 3D Skeletons

By Piotr Koniusz, Anoop Cherian, Fatih Murat Porikli. 2016

118 citations recorded by the source corpus.

In this paper, we explore tensor representations that can compactly capture higher-order relationships between skeleton joints for 3D action recognition. We first define RBF kernels on 3D joint sequences, which are then linearized to form kernel descriptors. The higher-order outer-products of these kernel descriptors form our tensor representations. We present two different kernels for action recognition, namely (i) a sequence compatibility kernel that captures the spatio-temporal compatibility of joints in one sequence against those in the other, and (ii) a dynamics compatibility kernel that explicitly models the action dynamics of a sequence. Tensors formed from these kernels are then used to train an SVM. We present experiments on several benchmark datasets and demonstrate state of the art results, substantiating the effectiveness of our representations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks (2017, background): “[51] introduced tensor representations for capturing the high-order relationships among”
Learning and Refining of Privileged Information-Based RNNs for Action Recognition from Depth Sequences (2017, methodology): “However, these approaches require either skeleton joints [61, 9, 22] or hand-crafted depth features [42] as inputs in both training and testing.”
A New Representation of Skeleton Sequences for 3D Action Recognition (2017, background): “In this paper, we focus on skeleton-based action recognition.”
Hallucinating IDT Descriptors and I3D Optical Flow Features for Action Recognition With CNNs (2019, background): “Another aggregation family captures second- or higher-order statistics [8, 32, 37, 16].”
Global Context-Aware Attention LSTM Networks for 3D Action Recognition (2017, background): “[22] explored tensor representations to capture the high-order relationships between the skeletal joints.”
Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition (2018, background): “[26] presented two kernel-based tensor representations to capture the compatibility between two action sequences and the dynamic information of a single action.”
Coding Kendall's Shape Trajectories for 3D Action Recognition (2018, background): “More recently, two kernel-based tensor representations named sequence compatibility kernel (SCK) and dynamics compatibility kernel (DCK) were introduced in [26].”
Domain Adaptation by Mixture of Alignments of Second-or Higher-Order Scatter Tensors (2016, methodology): “The above applications use a single tensor, while our goal is to use tensors as the domainand class-specific representations, similar to the sum-kernel approaches [21, 19, 20], and apply them to alignment tasks.”
Ordered Pooling of Optical Flow Sequences for Action Recognition (2017, methodology): “There have been extensions of this approach to also include semantics of the action, such as using Fisher vectors [22, 31], using bag of visual words model [20], combining with depth data [18], or using human pose [14].”
Spatio-Temporal Naive-Bayes Nearest-Neighbor (ST-NBNN) for Skeleton-Based Action Recognition (2017, methodology): “In addition, our method is better than the non-linear models [29, 8, 25], and the deep learning based methods [14, 5].”