Multiple object tracking: A literature review

By Wenhan Luo, Junliang Xing, Anton Milan, Xiaoqin Zhang, Wei Liu, Xiaowei Zhao, Tae-Kyun Kim. 2014

1,031 citations recorded by the source corpus.

Multiple Object Tracking (MOT) is an important computer vision problem which has gained increasing attention due to its academic and commercial potential. Although different kinds of approaches have been proposed to tackle this problem, it still remains challenging due to factors like abrupt appearance changes and severe object occlusions. In this work, we contribute the first comprehensive and most recent review on this problem. We inspect the recent advances in various aspects and propose some interesting directions for future research. To the best of our knowledge, there has not been any extensive review on this topic in the community. We endeavor to provide a thorough review on the development of this problem in recent decades. The main contributions of this review are fourfold: 1) Key aspects in a multiple object tracking system, including formulation, categorization, key principles, evaluation of an MOT are discussed. 2) Instead of enumerating individual works, we discuss existing approaches according to various aspects, in each of which methods are divided into different groups and each group is discussed in detail for the principles, advances and drawbacks. 3) We examine experiments of existing publications and summarize results on popular datasets to provide quantitative comparisons. We also point to some interesting discoveries by analyzing these results. 4) We provide a discussion about issues of MOT research, as well as some interesting directions which could possibly become potential research effort in the future.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “Object tracking is also often referred to as multiple object tracking (MOT) [180] and detection and tracking of multiple objects (DATMO) [181].”
Unsupervised Deep Tracking (2019, methodology): “The state-of-the-art deep tracking methods [1, 46, 15, 55, 27, 60, 58, 54, 4, 19, 33, 34] typically use pretrained CNN models for feature extraction.”
Vision-based vehicle detection and counting system using deep learning in highway scenes (2019, background): “Related work on vehicle tracking Advanced vehicle object detection applications, such as multi-object tracking, are also a critical ITS task [26].”
HOTA: A Higher Order Metric for Evaluating Multi-object Tracking (2020, methodology): “…for evaluation in the PETS workshop series [23] and remains, to this day, the most commonly used metric for evaluating MOT algorithms, although it has often been highly criticised [65,6,43,40,55,60,18,48,51,76, 52,88,20,50] for its bias toward overemphasizing detection over association (see Fig.”
Deep learning in video multi-object tracking: A survey (2019, background): “[11] presented the first comprehensive review to focus specifically on MOT, in particular on pedestrian tracking.”
Crossover Learning for Fast Online Video Instance Segmentation (2021, neutral): “For the definition of online and offline, we follow [33, 39].”
Video Instance Segmentation using Inter-Frame Communication Transformers (2021, background): “The models can be easily customized to real-world applications as they run in an online [9] fashion, but they show limitation in dealing with occlusions and motion blur that are common in videos.”
Rethinking the Competition Between Detection and ReID in Multiobject Tracking (2020, background): “Multi-object tracking (MOT), aiming to estimate the locations and scales of multiple targets in a video sequence, is one of the most fundamental yet challenging tasks in computer vision [1].”
Deep Affinity Network for Multiple Object Tracking (2018, background): “TRACKING multiple objects in videos [1] requires detection of objects in individual frames and associating those across multiple frames.”