Hierarchical Attention Network for Action Recognition in Videos

By Yilin Wang, Suhang Wang, Jiliang Tang, Neil O'Hare, Yi Chang, Baoxin Li. 2016

85 citations recorded by the source corpus.

Understanding human actions in wild videos is an important task with a broad range of applications. In this paper we propose a novel approach named Hierarchical Attention Network (HAN), which enables to incorporate static spatial information, short-term motion information and long-term video temporal structures for complex human action understanding. Compared to recent convolutional neural network based approaches, HAN has following advantages (1) HAN can efficiently capture video temporal structures in a longer range; (2) HAN is able to reveal temporal transitions between frame chunks with different time steps, i.e. it explicitly models the temporal transitions between frames as well as video segments and (3) with a multiple step spatial temporal attention mechanism, HAN automatically learns important regions in video frames and temporal segments in the video. The proposed model is trained and evaluated on the standard video action benchmarks, i.e., UCF-101 and HMDB-51, and it significantly outperforms the state-of-the arts

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks (2017, methodology): “Moreover, a recurrent attention mechanism is proposed to iteratively promote the attention capability of our network, while the methods in [60], [68], and [69] performed attention only once.”
Global Context-Aware Attention LSTM Networks for 3D Action Recognition (2017, background): “There are several works which explored attention, such as [39, 58], however, our method is significantly different from them in the following aspects: They all use the state of the previous time step of LSTM, whose contextual information is quite local, to provide the attention scores for the next t”
A General Survey on Attention Mechanisms in Deep Learning (2022, background): “For example, [46] proposes to use hierarchical attention in a video action recognition model to capture motion information at the the long-term level and the short-term level.”
SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction (2019, background): “Attention mechanisms have been proven to be signiﬁcantly effective for relevant data selection in various tasks [22,38, 41, 43].”
Spatiotemporal Multiplier Networks for Video Action Recognition (2017, unknown): “9% Hierarchical Attention Nets [40] 92.”