Online Multi-Target Tracking Using Recurrent Neural Networks

By Anton Milan, Seyed Hamid Rezatofighi, Anthony R. Dick, Ian David Reid, Konrad Schindler. AAAI Conference on Artificial Intelligence, 2016

537 citations recorded by the source corpus.

We present a novel approach to online multi-target tracking based on recurrent neural networks (RNNs). Tracking multiple objects in real-world scenes involves many challenges, including a) an a-priori unknown and time-varying number of targets, b) a continuous state estimation of all present targets, and c) a discrete combinatorial problem of data association. Most previous methods involve complex models that require tedious tuning of parameters. Here, we propose for the first time, an end-to-end learning approach for online multi-target tracking. Existing deep learning methods are not designed for the above challenges and cannot be trivially applied to the task. Our solution addresses all of the above points in a principled way. Experiments on both synthetic and real data show promising results obtained at ~300 Hz on a standard CPU, and pave the way towards future research in this direction.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/511bdaaa88788c71d0e79a43e4fa37d548db0cb9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Video Object Detection with an Aligned Spatial-Temporal Memory (2017, background): “o classiﬁcation [25,45,37,48,4]. These methods tend to model entire video frames instead of pixels, which means the ﬁne-grained details required for localizing objects are often lost. Object tracking [20,30,53, 36,34,43,28], which requires accurate localization, is also closely-related. The key diff”
Deep learning in video multi-object tracking: A survey (2019, methodology): “in [133], already described in section 3.”
An Introduction to Deep Visual Explanation (2017, background): “Over the last few years, deep learning algorithms have shown impressive results on variety of classification problems such as image classification [16,9], video action recognition[2], natural language processing [11,15], object tracking [8], image segmentations [12] and many others.”
Features for Multi-target Multi-camera Tracking and Re-identification (2018, background): “end prediction of trajectories [52] or by learning data association by back-propagating through a network-flow solution [60].”
Online Multi-Object Tracking with Dual Matching Attention Networks (2018, background): “Recently, several on-line approaches [10,27,36,42,2] using deep learning models have been proposed.”
Spatial-Temporal Relation Networks for Multi-Object Tracking (2019, methodology): “Besides, aggregation information across frames is also verified to be beneficial for measuring similarity [46, 26, 35].”
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking (2020, methodology): “to-end manner. On MOT16, this method attains MOTA of 50.31%. We do observe trends towards learning to perform end-to-end MOT. To the best of our knowledge, the rst method attempting this is RNN LSTM [Milan et al., 2017], which jointly learns motion anity costs and to perform bi-partite detection ass”
Deep Affinity Network for Multiple Object Tracking (2018, unknown): “8 RNN_LSTM [91] online 13.”
Artificial intelligence in the creative industries: a review (2020, methodology): “Recurrent networks have been integrated with object recognition methods to track the detected objects over time in [186–188].”
Quasi-Dense Similarity Learning for Multiple Object Tracking (2020, methodology): “Appearance similarity in MOT To exploit instance appearance similarity to strengthen tracking and re-identify vanished objects, some methods directly use an independent model [2,20,22,29,38,40,45,49] or add an extra embedding head to the detector for end-to-end training [26, 44, 50, 53].”