A Biologically Inspired System for Action Recognition

By Hueihan Jhuang, Thomas Serre, Lior Wolf, Tomaso A. Poggio. 2007

821 citations recorded by the source corpus.

We present a biologically-motivated system for the recognition of actions from video sequences. The approach builds on recent work on object recognition based on hierarchical feedforward architectures [25, 16, 20] and extends a neurobiological model of motion processing in the visual cortex [10]. The system consists of a hierarchy of spatio-temporal feature detectors of increasing complexity: an input sequence is first analyzed by an array of motion- direction sensitive units which, through a hierarchy of processing stages, lead to position-invariant spatio-temporal feature detectors. We experiment with different types of motion-direction sensitive units as well as different system architectures. As in [16], we find that sparse features in intermediate stages outperform dense ones and that using a simple feature selection approach leads to an efficient system that performs better with far fewer features. We test the approach on different publicly available action datasets, in all cases achieving the highest results reported to date.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Neuroscience

Citations in the corpus

RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, methodology): “[60] used a feedforward hierarchical template matching architecture for action recognition with pre-defined spatio-temporal filters in the first layer.”
Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks (2015, neutral): “1% Spatio-temporal HMAX network [6] [14] -% 22.”
Unstructured human activity detection from RGBD images (2011, methodology): “We have also made the dataset and code available open-source at: http://pr.cs.cornell.edu/humanactivities”
Pose primitive based human action recognition in videos or still images (2008, background): “[10] introduces a biologically inspired action recognition approach which uses hierarchically ordered spatio-temporal feature detectors.”
Action snippets: How many frames does human action recognition require? (2008, methodology): “This size – determined experimentally – agrees with the findings of [17, 26].”
Motion Interchange Patterns for Action Recognition in Unconstrained Videos (2012, methodology): “Baseline results, taken from [31], include the combination of the HOG and HOF STIP systems [30], as well as the biologicallymotivated Action Recognition system [32] based on a model of the human visual”
Learning deformable action templates from cluttered videos (2009, methodology): “This is slightly different from previous methods used in [6, 17], where optical flow maps are computed according to grids in both direction and speed.”
Characteristic Kernels on Structured Domains Excel in Robotics and Human Action Recognition (2010, background): “Now we confirm theoretical advantages of characteristic kernels on various practical applications.”
Recognize Human Activities from Partially Observed Videos (2013, methodology): “We choose several state-of-the-art comparison methods, including Ryoo’s human activity prediction methods (both non-dynamic and dynamic versions) [ 14], early event detector – MMED [6], C2 [7], and Action Bank [ 17].”
Dynamic Scene Classification: Learning Motion Descriptors with Slow Features Analysis (2013, methodology): “Some models with biological inspiration also use this coding and pooling approach [12, 18].”
HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition (2017, methodology): “Other methods started to explore the integration of dynamical information into recognition by using motion-direction sensitive units [19] or dynamical networks (like Echo-state networks) [20].”