The THUMOS challenge on action recognition for videos "in the wild"

By Haroon Idrees, Amir Zamir, Yu-Gang Jiang, Alexander N. Gorban, Ivan Laptev, Rahul Sukthankar, Mubarak Shah. Computer Vision and Image Understanding, 2016

902 citations recorded by the source corpus.

Abstract Automatically recognizing and localizing wide ranges of human actions are crucial for video understanding. Towards this goal, the THUMOS challenge was introduced in 2013 to serve as a benchmark for action recognition. Until then, video action recognition, including THUMOS challenge, had focused primarily on the classification of pre-segmented (i.e., trimmed) videos, which is an artificial task. In THUMOS 2014, we elevated action recognition to a more practical level by introducing temporally untrimmed videos. These also include ‘background videos’ which share similar scenes and backgrounds as action videos, but are devoid of the specific actions. The three editions of the challenge organized in 2013–2015 have made THUMOS a common benchmark for action classification and detection and the annual challenge is widely attended by teams from around the world. In this paper we describe the THUMOS benchmark in detail and give an overview of data collection and annotation procedures. We present the evaluation protocols used to quantify results in the two THUMOS tasks of action classification and temporal action detection. We also present results of submissions to the THUMOS 2015 challenge and review the participating approaches. Additionally, we include a comprehensive empirical study evaluating the differences in action recognition between trimmed and untrimmed videos, and how well methods trained on trimmed videos generalize to untrimmed videos. We conclude by proposing several directions and improvements for future THUMOS challenges.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3dd906bc0947e56d2b7bf9530b11351bbdff2358

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Online action detection (2017, methodology): “Later, many large-scale datasets with more action diversity were introduced, such as THUMOS [9], MEXAction2 [1] and ActivityNet [4].”
Attend and Interact: Higher-Order Object Interactions for Video Understanding (2017, background): “Although recent state-of-the-art approaches for action recognition have demonstrated significant improvements over datasets such as UCF101 [45], HMDB51 [23], Sports1M [20], THUMOS [18], ActivityNet [5], and YouTube8M [1], they often focus on representing the overall visual scene (coarse-grained) as ”
Temporal Segment Networks for Action Recognition in Videos (2017, methodology): “We perform experiments on five challenging action recognition datasets, namely HMDB51 [29], UCF101 [28], THUMOS [26], ActivityNet [27], and Kinetics [30], to verify the effectiveness of our method for action recognition in both trimmed and untrimmed videos.”
ActionFormer: Localizing Moments of Actions with Transformers (2022, background): “THUMOS14 [29] dataset contains 413 untrimmed videos with 20 categories of actions.”
3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization (2019, methodology): “We perform comprehensive experiments on two benchmarks: THUMOS14 [9] and ActivityNet 1.”
BABEL: Bodies, Action and Behavior with English Labels (2021, neutral): “Many works over the years have contributed to progress in the action localization task [13, 29, 38].”
Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization (2019, methodology): “On two benchmark datasets THUMOS’14 [21] and ActivityNet [6], the proposed method outperforms the stateof-the-art methods, demonstrating the effectiveness of handling the two problems.”
W-TALC: Weakly-supervised Temporal Activity Localization and Classification (2018, methodology): “We use mAP with different Intersection over Union (IoU) thresholds as a performance metric, as in [21].”
A Review on Deep Learning Techniques for Video Prediction (2020, background): “MCnet [65] 2017 [13], [166], [202] LSTM-cED [111], [112], [115], [123] RGB RGB X `p, GDL,AL R ? ? ? 7 X Dual-GAN [55] 2017 [100] VAE-GAN [115], [118]–[120] RGB RGB X `1,KL,AL R ? ? 7 7 DRNET [79] 2017 [65] LSTM-ED [74], [111], [138], [203] RGB RGB X `2, CE,AL SR ? ? ? ? X X”
Speech2Action: Cross-Modal Supervision for Action Recognition (2020, methodology): “Evaluation is performed using average classification accuracy over three train/test splits from [16], each with 3,570 train and 1,530 test videos.”
Application of Motion-Based Visual Servoing to Target Tracking (2001, methodology): “We get the following from the classical equation between image point velocity and 3-D kinematic screw (Espiau, Chaumette, and Rives 1992; Hutchinson, Hager, and Corke 1996; Crétual and Chaumette 2001):”
Timeception for Complex Action Recognition (2018, background): “MultiTHUMOS extends the original THUMOS-14 [53] by providing multilabel annotation for the videos in validation and test splits.”