Hierarchical Attentive Recurrent Tracking

By Adam R. Kosiorek, Alex Bewley, Ingmar Posner. 2017

63 citations recorded by the source corpus.

Class-agnostic object tracking is particularly difficult in cluttered environments as target specific discriminative models cannot be learned a priori. Inspired by how the human visual cortex employs spatial attention and separate "where" and "what" processing pathways to actively suppress irrelevant visual features, this work develops a hierarchical attentive recurrent model for single object tracking in videos. The first layer of attention discards the majority of background by selecting a region containing the object of interest, while the subsequent layers tune in on visual features particular to the tracked object. This framework is fully differentiable and can be trained in a purely data driven fashion by gradient methods. To improve training convergence, we augment the loss function with terms for a number of auxiliary tasks relevant for tracking. Evaluation of the proposed model is performed on two datasets: pedestrian tracking on the KTH activity recognition dataset and the more difficult KITTI object tracking dataset.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering, Global and Planetary Change

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Global and Planetary Change

Citations in the corpus

On the Binding Problem in Artificial Neural Networks (2020, methodology): “, 2015), instance segmentation (Ren and Zemel, 2017), or (multi-)object tracking (Kosiorek et al., 2017; Fuchs et al., 2019).”
Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects (2018, methodology): “S QAIR is unsupervised and hence it does not rely on bounding boxes nor additional labels for training, while being able to learn arbitrary motion and appearance models similarly to HART (Kosiorek et al., 2017).”
Towards Interpretable Reinforcement Learning Using Attention Augmented Agents (2019, background): “image classification and captioning [12, 13, 14, 15, 16, 17, 18, 19, 20], text classification [21, 22], generative models [6, 23, 24], object tracking [25], and reinforcement learning [26, 27, 28].”
SCALOR: Generative World Models with Scalable Object Representations (2019, methodology): “Furthermore, there is a substantial amount of work on object tracking from the computer vision community using the same “bounding box”-representation approach proposed in SCALOR (Kosiorek et al., 2017; Ning et al., 2017; Nam & Han, 2016; Tao et al., 2016), but such methods use provided labels to tac”
Visual Attention Methods in Deep Learning: An In-Depth Survey (2022, methodology): “tiple computer vision tasks, either employing a different attention map for every image pixel, comparing it with the representations of other pixels [3], [4], [9] or generating an attention map to extract the global representation for the whole image [10], [11].”
End-to-end Recurrent Multi-Object Tracking and Trajectory Prediction with Relational Reasoning (2019, methodology): “We start by describing the hierarchical attentive recurrent tracking (HART) algorithm [6], and then follow with an extension of HART to tracking multiple objects, where multiple instances of HART communicate with each other using multi-headed attention to facilitate relational reasoning.”
Towards Robust Image Classification Using Sequential Attention Models (2019, methodology): “Recurrent attention models: Attention mechanisms have been widely used in many sequence modeling problems such as question-answering [23], machine translation [6, 51], video classification and captioning [45, 32], image classification and captioning [36, 11, 17, 1, 54, 59, 52, 5, 56], text classific”
A Twofold Siamese Network for Real-Time Object Tracking (2018, methodology): “HART [18] and RATM[16] use RNN with attention model to separate where and what processing pathways to actively suppress irrelevant visual features.”
Scalable Object-Oriented Sequential Generative Models (2019, methodology): “Overall, there has numerous amount of works on object tracking from the computer vision community using the same “bounding box”-representation approach proposed in SCALOR (Kosiorek et al., 2017; Ning et al., 2017; Nam & Han, 2016; Tao et al., 2016), but such methods use provided labels to tackles th”