Learning Activity Progression in LSTMs for Activity Detection and Early Detection

By Shugao Ma, Leonid Sigal, Stan Sclaroff. 2016

392 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Time-Series & Forecasting

Citations in the corpus

RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, neutral): “Ma et al. (2016) introduced a novel ranking loss Fig.”
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks (2017, methodology): “They have been successfully applied to language modeling [18], RGB based video analysis [19]–[27], and also skeleton based action recognition [12], [28], [29].”
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates (2017, neutral): “[25], [26], [27], [28], [29], [30], and RGB-based activity recognition [31], [32], [33], [34], [35], [36].”
TALL: Temporal Activity Localization via Language Query (2017, methodology): “For traditional temporal action localization, most current approaches [30, 15, 26, 34, 35] apply activity classifiers trained with optical flow-based methods [33, 28] or Convolutional Neural Networks (CNNs) [29, 32] in a sliding window fashion.”
Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video (2020, methodology): “LSTM has been trained using cross entropy, RL has been trained using the ranking loss on the detection score proposed in [7], whereas EL ihas been trained using the exponential anticipation loss proposed in [14].”
Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization (2017, background): “Other researchers have used language models [32], action progression analysis [26,34], and high-level semantics [5] to produce high-fidelity action detections.”
Encouraging LSTMs to Anticipate Actions Very Early (2017, methodology): “DP-SVM [32] 5% 46% S-SVM [32] 5% 43% Where/What [33] 10% 43% Ranking Loss [21] 29% 43% Context-Aware+Loss of [11] 28 % 43% Context-Aware+Loss of [21] 33% 39%”
Leveraging Uncertainty to Rethink Loss Functions and Evaluation Measures for Egocentric Action Anticipation (2018, background): “[22] address early action detection constraining predicted action scores to increase monotonically over time.”
Progressive Teacher-Student Learning for Early Action Prediction (2019, methodology): “35%, outperforming all of the competitors including STUDENT, RankLSTM [28], DeepSCN [24], and MSRNN [16].”
Temporal Context Network for Activity Localization in Videos (2017, background): “Very recently, studies focusing on temporal segments which contain human actions have been introduced [18, 4, 29, 17, 31].”
Cascaded Boundary Regression for Temporal Action Detection (2017, background): “[14] addressed the problem of early action detection.”
AdaFrame: Adaptive Frame Selection for Fast Video Recognition (2018, methodology): “Inspired by [12], we introduce a reward function that forces the classifier to be more confident when seeing additional frames, taking the following form (when t > 1):”
Spatial, Structural and Temporal Feature Learning for Human Interaction Prediction (2016, methodology): “Recently, LSTM has also been used for activity detection [22], video recognition and description [23].”
Global Context-Aware Attention LSTM Networks for 3D Action Recognition (2017, background): “Recently, Long Short-Term Memory (LSTM) network [15] has been successfully applied to language modeling [46], RGB-based activity analysis [17, 68, 69, 61, 10, 21, 43, 30], and also 3D action recognition [11, 73, 27] due to its strong power in modeling sequential data.”
Peeking Into the Future: Predicting Future Person Activities and Locations in Videos (2019, background): “[20] and [2] proposed different losses to encourage LSTM to recognize actions early in internet videos.”
Deep Sequential Context Networks for Action Prediction (2017, background): “By comparison, [35, 21, 14] predict what will happen in the future, and [9, 19] localizes the starting and ending frames of an incomplete event.”
Rethinking the Faster R-CNN Architecture for Temporal Action Localization (2018, background): “Another line of work generates frame-wise or snippetwise action labels, and uses these labels to define the temporal boundaries of actions [29, 38, 10, 26, 55, 20].”