A spatiotemporal model with visual attention for video classification

By Mo Shan, Nikolay A. Atanasov. 2017

11 citations recorded by the source corpus.

High level understanding of sequential visual input is important for safe and stable autonomy, especially in localization and object detection. While traditional object classification and tracking approaches are specifically designed to handle variations in rotation and scale, current state-of-the-art approaches based on deep learning achieve better performance. This paper focuses on developing a spatiotemporal model to handle videos containing moving objects with rotation and scale changes. Built on models that combine Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to classify sequential data, this work investigates the effectiveness of incorporating attention modules in the CNN stage for video classification. The superiority of the proposed spatiotemporal model is demonstrated on the Moving MNIST dataset augmented with rotation and scaling.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Interpretable Reinforcement Learning Using Attention Augmented Agents (2019, neutral): “They have been applied with some success to question-answering datasets [7, 8], text translation [5, 9], video classification and captioning [10, 11],”
Towards Robust Image Classification Using Sequential Attention Models (2019, background): “…widely used in many sequence modeling problems such as question-answering [24], machine translation [6, 52], video classiﬁcation and captioning [46, 33], image classiﬁcation and captioning [37, 11, 17, 1, 55, 60, 53, 5, 57], text classiﬁcation [58, 47], generative models [42, 59, 30], object…”