Efficient Action Detection in Untrimmed Videos via Multi-task Learning

By Yi Zhu, Shawn D. Newsam. 2016

34 citations recorded by the source corpus.

This paper studies the joint learning of action recognition and temporal localization in long, untrimmed videos. We employ a multi-task learning framework that performs the three highly related steps of action proposal, action recognition, and action localization refinement in parallel instead of the standard sequential pipeline that performs the steps in order. We develop a novel temporal actionness regression module that estimates what proportion of a clip contains action. We use it for temporal localization but it could have other applications like video retrieval, surveillance, summarization, etc. We also introduce random shear augmentation during training to simulate viewpoint change. We evaluate our framework on three popular video benchmarks. Results demonstrate that our joint model is efficient in terms of storage and computation in that we do not need to compute and cache dense trajectory features, and that it is several times faster than its sequential ConvNets counterpart. Yet, despite being more efficient, it outperforms stateof-the-art methods with respect to accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, background): “Also, rather than using one fixed scale [24] or multiscale multi-round scans [70], [71], we propose a novel SSNet for online prediction, which is supervised to choose the”
SSNet: Scale Selection Network for Online 3D Action Prediction (2018, background): “Also, instead of using one fixed scale [77] or multi-scale multi-pass scans [55, 84], we propose a novel SSNet for online prediction, which is supervised to choose the proper window for prediction at each temporal step.”
DenseNet for dense flow (2017, background): “Another concern of this work is that DenseNet has a large memory bandwidth which may limit its potential applications like action recognition [18, 19, 20].”