Diagnosing Error in Temporal Action Detectors

By Humam Alwassel, Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem. 2018

123 citations recorded by the source corpus.

Despite the recent progress in video understanding and the continuous rate of improvement in temporal action localization throughout the years, it is still unclear how far (or close?) we are to solving the problem. To this end, we introduce a new diagnostic tool to analyze the performance of temporal action detectors in videos and compare different methods beyond a single scalar metric. We exemplify the use of our tool by analyzing the performance of the top rewarded entries in the latest ActivityNet action localization challenge. Our analysis shows that the most impactful areas to work on are: strategies to better handle temporal context around the instances, improving the robustness w.r.t. the instance absolute and relative size, and strategies to reduce the localization errors. Moreover, our experimental analysis finds the lack of agreement among annotator is not a major roadblock to attain progress in the field. Our diagnostic tool is publicly available to keep fueling the minds of other researchers with additional insights about their algorithms.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[9] Humam Alwassel, Fabian Caba Heilbron, Victor Escorcia, and Bernard Ghanem.”
End-to-End Temporal Action Detection With Transformer (2021, background): “Besides, the boundaries of actions are less clear than those of objects [18].”
Temporal Localization of Moments in Video Collections with Natural Language (2019, background): “Also note that humans may not generally agree on temporal endpoints [1].”
Learning Salient Boundary Feature for Anchor-free Temporal Action Localization (2021, background): “Additionally, compared with THUMOS14, ActivityNet is less well annotated as explained by official report [1] showing ‘it is hard to agree about the temporal boundaries’, which partially attributes”
ActionFormer: Localizing Moments of Actions with Transformers (2022, methodology): “These analyses are obtained using the tool provided by [1].”
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets (2020, background): “More recently, tools in the video domain (Alwassel et al., 2018) are looking into forms of dataset bias in activity recognition (Sigurdsson et al., 2017).”
Video Self-Stitching Graph Network for Temporal Action Localization (2020, methodology): “The accumulated mAP considers the number of action instances N i and the normalized mAP [1] within each action-length group, which is formulated as mAP acc = L (120, 180], and XL (180, inf).”