Slow Feature Analysis for Human Action Recognition

By Zhang Zhang, Dacheng Tao. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012

324 citations recorded by the source corpus.

Slow Feature Analysis (SFA) extracts slowly varying features from a quickly varying input signal [1]. It has been successfully applied to modeling the visual receptive fields of the cortical neurons. Sufficient experimental results in neuroscience suggest that the temporal slowness principle is a general learning principle in visual perception. In this paper, we introduce the SFA framework to the problem of human action recognition by incorporating the discriminative information with SFA learning and considering the spatial relationship of body parts. In particular, we consider four kinds of SFA learning strategies, including the original unsupervised SFA (U-SFA), the supervised SFA (S-SFA), the discriminative SFA (D-SFA), and the spatial discriminative SFA (SD--SFA), to extract slow feature functions from a large amount of training cuboids which are obtained by random sampling in motion boundaries. Afterward, to represent action sequences, the squared first order temporal derivatives are accumulated over all transformed cuboids into one feature vector, which is termed the Accumulated Squared Derivative (ASD) feature. The ASD feature encodes the statistical distribution of slow features in an action sequence. Finally, a linear support vector machine (SVM) is trained to classify actions represented by ASD features. We conduct extensive experiments, including two sets of control experiments, two sets of large scale experiments on the KTH and Weizmann databases, and two sets of experiments on the CASIA and UT-interaction databases, to demonstrate the effectiveness of SFA for human action recognition. Experimental results suggest that the SFA-based approach (1) is able to extract useful motion patterns and improves the recognition performance, (2) requires less intermediate processing steps but achieves comparable or even better performance, and (3) has good potential to recognize complex multiperson activities.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/3a6b1896c5d76b2c85467cb0611104d8f9de2b87

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

DL-SFA: Deeply-Learned Slow Feature Analysis for Action Recognition (2014, background): “As the authors extracted the slow features from the input data directly and only once, [35] can be somehow thought of as one-layer SFA, a special case of our proposed DL-SFA.”
Deep Feature Flow for Video Recognition (2016, methodology): “This observation has been used to regularize the feature learning in videos [45, 21, 51, 49, 40].”
Effective 3D action recognition using EigenJoints (2014, background): “Traditional research mainly concentrates on action recognition of video sequences captured by RGB cameras [2,3,6,9,10,12,30].”
Human Action Recognition With Video Data: Research and Evaluation Challenges (2014, background): “and slow feature analysis [151] do not require extensive training since they can learn and update the feature representation models based on new training samples.”
A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior (2015, background): “Therefore, improving the technique of image haze removal will benefit many image understanding and computer vision applications such as aerial imagery [1], image classification [2]–[5], image/video retrieval [6]–[8], remote sensing [9]–[11] and video analysis and recognition [12]–[14].”
Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks (2015, methodology): “To recognize human actions in video sequences, computer vision researchers have been developing better visual features to characterize the spatial appearance [32, 20] and temporal motion [16, 24, 35].”
Slow Feature Analysis for Change Detection in Multispectral Imagery (2014, methodology): “SFA has been successfully applied to invariant object recognition in spatial transformation, nonlinear blind source separation, and human action recognition [47]–[49].”
Slow and Steady Feature Analysis: Higher Order Temporal Coherence in Video (2015, background): “Building on SFA subspace ideas [42], researchers have also examined slow features for action recognition [45], facial expression analysis [44], future prediction [39], and temporal segmentation [31, 27].”
Online human gesture recognition from motion data streams (2013, unknown): “Most existing approaches [10, 14, 33, 35, 37] are about”
How to solve classification and regression problems on high-dimensional data with a supervised extension of slow feature analysis (2013, methodology): “The equivalence of the discrimination capability of SFA and FDA in some setups was already known (compare Berkes, 2005a, and Berkes, 2005b) but had not been rigorously shown before.”
Unsupervised Learning using Sequential Verification for Action Recognition (2016, background): “[59], in particular, show how such constraints are useful for action recognition.”
Progressive Teacher-Student Learning for Early Action Prediction (2019, background): “Some representative handcrafted features like Cuboids [7] [44] [6], interest point clouds [4], 3DHOG [20], SIFT [33], and dense trajectory [41] etc.”
Unsupervised Deep Slow Feature Analysis for Change Detection in Multi-Temporal Remote Sensing Images (2018, unknown): “And it has been successfully applied to solve diverse real-world problems, such as human action recognition, dynamic texture recognition and time series analysis, etc [32]–[35].”
An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos (2018, background): “SFA has been well known in pattern recognition and has been applied to the problem of activity recognition [64], [65].”
Dynamic Scene Classification: Learning Motion Descriptors with Slow Features Analysis (2013, methodology): “Scenes HOF [23, 9] GIST [26, 9] Chaos [29] SOE [9] [35] SFA Embedding No SFA Our Model Avalange 0 10 30 10 80 90 90 Boiling.”
Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification (2016, background): “[44], in particular, show how such constraints are useful for action recognition.”
Web-Based Classifiers for Human Action Recognition (2012, background): “While it is known that biologically inspired features are also effective for action recognition [4], [47], most of these features require uncluttered backgrounds and/or accompanying motion information.”