Going deeper into action recognition: A survey

By Samitha Herath, Mehrtash Harandi, Fatih Murat Porikli. 2016

642 citations recorded by the source corpus.

Understanding human actions in visual data is tied to advances in complementary research areas including object recognition, human dynamics, domain adaptation and semantic segmentation. Over the last decade, human action analysis evolved from earlier schemes that are often limited to controlled environments to nowadays advanced solutions that can learn from millions of videos and apply to almost all daily activities. Given the broad range of applications from video surveillance to human-computer interaction, scientific milestones in action recognition are achieved more rapidly, eventually leading to the demise of what used to be good in a short time. This motivated us to provide a comprehensive review of the notable steps taken towards recognizing human actions. To this end, we start our discussion with the pioneering methods that use handcrafted representations, and then, navigate into the realm of deep learning based approaches. We aim to remain objective throughout this survey, touching upon encouraging improvements as well as inevitable fallbacks, in the hope of raising fresh questions and motivating new research directions for the reader.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, background): “Since an extensive literature review is prohibitive here due to the limited size of the paper, we encourage the readers to refer to the surveys in [43, 19] for respectively pose estimation and action recognition.”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, background): “This challenge remains even with current deep learning approaches (Asadi-Aghbolaghi et al., 2017; Herath et al., 2017).”
Interpretable 3D Human Action Analysis with Temporal Convolutional Networks (2017, background): “Traditionally, the community has focused on activity recognition in the domain of RGB videos [34, 13].”
Learning a Generative Model for Multi‐Step Human‐Object Interactions from Videos (2019, positive): “Human Object Interaction Perception : In the action recognition literature [HHP17], methods aim to encode the complexity of human activity spaces and the interactions therein.”
SST: Single-Stream Temporal Action Proposals (2017, background): “A large body of research has tackled the problem of action categorization from short video clips [14].”
Action Recognition From Single Timestamp Supervision in Untrimmed Videos (2019, background): “For a review of works that use strong supervision, we refer the reader to [13].”
Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization (2019, background): “Please refer to recent surveys [1, 3, 22, 19] for a detailed review.”
Vision-based human activity recognition: a survey (2020, background): “A comparative analysis of methods based on handcrafted representations and solutions that involve learning architectures is carried out in [68] and [236].”
Vision-Language Models for Vision Tasks: A Survey (2023, background): “Image classification [8], [9] and action recognition [107], [108] have been widely adopted in such evaluations, where video clips are often sub-sampled for efficient recognition in action recognition tasks [14].”
BABEL: Bodies, Action and Behavior with English Labels (2021, neutral): “Relevant tasks include predicting semantic labels for a human movement, e.g., action recognition [14], video description [39], temporal localiza-* Denotes equal contribution.”
Domain Adaptation by Mixture of Alignments of Second-or Higher-Order Scatter Tensors (2016, methodology): “a source represented by a CNN trained on images [7, 22] and a target using an LSTM [16, 15] which is trained on text corpora or video data [13].”