Learning a Deep Model for Human Action Recognition from Novel Viewpoints

By Hossein Rahmani, Ajmal Saeed Mian, Mubarak Shah. 2016

220 citations recorded by the source corpus.

Recognizing human actions from unknown and unseen (novel) views is a challenging problem. We propose a Robust Non-Linear Knowledge Transfer Model (R-NKTM) for human action recognition from novel views. The proposed R-NKTM is a deep fully-connected neural network that transfers knowledge of human actions from any unknown view to a shared high-level virtual view by finding a set of non-linear transformations that connects the views. The R-NKTM is learned from 2D projections of dense trajectories of synthetic 3D human models fitted to real motion capture data and generalizes to real videos of human actions. The strength of our technique is that we learn a single R-NKTM for all actions and all viewpoints for knowledge transfer of any real human action video without the need for re-training or fine-tuning the model. Thus, R-NKTM can efficiently scale to incorporate new action classes. R-NKTM is learned with dummy labels and does not require knowledge of the camera viewpoint at any stage. Experiments on three benchmark cross-view human action datasets show that our method outperforms existing state-of-the-art.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, background): “Rahmani and Mian [33], [34] introduced a nonlinear knowledge transfer model to transform different views of human actions to a canonical view.”
Computer Vision and Image Understanding Computer Vision for Assistive Technologies (unknown): “In this field the state of the art is represented by the approach proposed in Rahmani et al. (2016) where a deep fully-connected neural network that transfers knowledge of human actions from any unknown view to a shared high-level virtual view by finding a non-linear virtual path that connects the…”
3D Action Recognition from Novel Viewpoints (2016, methodology): “Experiments on two multiview and three single-view benchmark datasets show that the proposed method dramatically outperforms existing state-of-the-art in action recognition.”
Deep Affinity Network for Multiple Object Tracking (2018, background): “s an essential component of various vision applications, e.g. robotics, surveillance; while it also provides footholds to several high-level vision tasks, e.g. people counting [5], action recognition [6], and pose estimation [7]. Simultaneous tracking of multiple objects in videos is a comprehensive”
Else-Net: Elastic Semantic Network for Continual Action Recognition from Skeleton Data (2021, background): “Various skeleton-based action recognition approaches [5, 29, 13, 17, 20, 18, 19, 30, 42, 15, 27, 31, 39, 14] have been proposed.”
Learning Clip Representations for Skeleton-Based 3D Action Recognition (2018, background): “Nowadays due to the prevalence of highly-accurate and affordable depth devices, there are more and more works using depth videos for computer vision tasks [2]–[5].”