Actionness Estimation Using Hybrid Fully Convolutional Networks

By Limin Wang, Yu Qiao, Xiaoou Tang, Luc Van Gool. 2016

104 citations recorded by the source corpus.

Actionness was introduced to quantify the likelihood of containing a generic action instance at a specific location. Accurate and efficient estimation of actionness is important in video analysis and may benefit other relevant tasks such as action recognition and action detection. This paper presents a new deep architecture for actionness estimation, called hybrid fully convolutional network (HFCN), which is composed of appearance FCN (A-FCN) and motion FCN (M-FCN). These two FCNs leverage the strong capacity of deep models to estimate actionness maps from the perspectives of static appearance and dynamic motion, respectively. In addition, the fully convolutional nature of H-FCN allows it to efficiently process videos with arbitrary sizes. Experiments are conducted on the challenging datasets of Stanford40, UCF Sports, and JHMDB to verify the effectiveness of H-FCN on actionness estimation, which demonstrate that our method achieves superior performance to previous ones. Moreover, we apply the estimated actionness maps on action proposal generation and action detection. Our actionness maps advance the current state-of-the-art performance of these tasks substantially.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection (2017, methodology): “The flow and magnitude values in the image are multiplied by 16 and quantized into the [0,255] interval [20, 31, 46, 47].”
Weakly Supervised Action Localization by Sparse Temporal Pooling Network (2017, methodology): “To facilitate action detection and localization, many algorithms use action proposals [4, 7, 38], which is an extension of object proposals for object detection in images.”
Online Real-Time Multiple Spatiotemporal Action Localisation and Prediction (2016, methodology): “Spatial action localisation is typically addressed using segmentation [23, 40, 11] or region proposal and actionness [7, 51] -based approaches.”
AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions (2017, methodology): “Our 3D two-stream model obtains stateFrame-mAP JHMDB UCF101-24 Actionness [42] 39.”
Action Tubelet Detector for Spatio-Temporal Action Localization (2017, methodology): “More recently, some approaches [17, 34, 38] rely on an actionness measure [4], i.”
TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals (2017, unknown): “Spatio-temporal action detection [31, 28] used Recall vs.”
UntrimmedNets for Weakly Supervised Action Recognition and Detection (2017, methodology): “In addition, several deep learning methods have been proposed for action proposal generation and detection [12, 47, 28, 10, 41, 37].”
Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos (2017, background): “Compared to action recognition, action detection is a more challenging problem [2, 7, 29], which has been an active area of research.”
R-C3D: Region Convolutional 3D Network for Temporal Activity Detection (2017, background): “Existing state-of-the-art approaches address this task as detection by classiﬁcation , i.e. classifying temporal segments generated in the form of sliding windows [13, 20, 24, 37] or via an external “pro-posal” generation mechanism [10, 35].”
AMTnet: Action-Micro-Tube Regression by End-to-end Trainable Deep Architecture (2017, methodology): “While many works concern either spatial action localisation [18, 32, 10, 25] in trimmed videos or temporal localisation [17, 3, 27, 19, 31, 23, 35, 34] in untrimmed videos, only a handful number of methods have been proposed to tackle both problems jointly.”
Encouraging LSTMs to Anticipate Actions Very Early (2017, background): “This is in contrast to existing methods that typically extract either global representations for the entire image [6, 41, 42, 7] or video sequence [35, 14], thus not focusing on the action itself, or localize the feature extraction process to the action itself via dense trajectories [39, 38, 9], opt”
In the Eye of the Beholder: Gaze and Actions in First Person Video (2020, background): “For example, several previous works on actionness map [62], [63] aimed at estimating regions of actions from raw video frames.”
A Survey on Deep Learning Based Approaches for Action and Gesture Recognition in Image Sequences (2017, background): “[113] simply estimates actionness maps from appearance and motion cues.”