MOT16: A Benchmark for Multi-Object Tracking

By Anton Milan, L. Leal-Taix'e, Ian David Reid, Stefan Roth, Konrad Schindler. 2016

2,130 citations recorded by the source corpus.

Standardized benchmarks are crucial for the majority of computer vision applications. Although leaderboards and ranking tables should not be over-claimed, benchmarks often provide the most objective measure of performance and are therefore important guides for reseach. Recently, a new benchmark for Multiple Object Tracking, MOTChallenge, was launched with the goal of collecting existing and new data and creating a framework for the standardized evaluation of multiple object tracking methods. The first release of the benchmark focuses on multiple people tracking, since pedestrians are by far the most studied object in the tracking community. This paper accompanies a new release of the MOTChallenge benchmark. Unlike the initial release, all videos of MOT16 have been carefully annotated following a consistent protocol. Moreover, it not only offers a significant increase in the number of labeled boxes, but also provides multiple object classes beside pedestrians and the level of visibility for every single object of interest.

Subfields: Computer Vision and Pattern Recognition, Global and Planetary Change

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Global and Planetary Change

Citations in the corpus

Deep Perm-Set Net: Learn to predict sets with unknown permutation and cardinality using deep neural networks (2018, methodology): “We use training sequences from MOTChallenge pedestrian detection and tracking benchmark, i.e. 2DMOT2015 Leal-Taixé et al. (2015) and MOT17Det Milan et al. (2016), to create our dataset.”
DeepSetNet: Predicting Sets with Deep Neural Networks (2016, methodology): “To this end, we perform experiments on two widely used datasets, Caltech Pedestrians [8] and MOT16 from the MOTChallenge benchmark [29].”
Occluded Video Instance Segmentation: A Benchmark (2021, background): “Some popular datasets focus on the tracking of pedestrians and cars in street scenes, such as MOT16 [56] and KITTI [22].”
Simitate: A Hybrid Imitation Learning Benchmark (2019, neutral): “many research topics like autonomous driving [14], RGB-D SLAM systems [15], object tracking [13], [37], [38].”
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning (2018, background): “…be feasible to label a large number of images with simple annotations such as drivable areas and object bounding boxes [11, 19], it remains challenging to obtain more complicated annotations such as instance segmentation [3], not to mention multi-object detection and segmentation tracking [31, 22].”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, methodology): “and multiperson pose tracking. Pose estimation is evaluated by the mean Average Precision (mAP) as done in [51,28]. Multiperson pose tracking is evaluated by the multi-object tracking accuracy (MOTA) [38,28]. Details are given in [28]. Training. We train our HRNet-W48 for single person pose estimati”
Video Instance Segmentation (2019, background): “It should be noted that there is some minor difference between our task and the multi-object tracking problem [18] in that a still object instance is treated as a ground truth, and if an object is occluded or out of scene for several frames then reappears in the following frames, the instance label ”
Tracking Objects as Points (2020, methodology): “ is sensitive to the task-dependent output threshold [46]. The threshold we use are listed in Section 5.2. The interplay between output threshold and true positive criteria matters. For 2D tracking [12, 28], >0:5bounding box IoU is a the true positive. For 3D tracking [3], bounding box center di”
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild (2018, background): “A number of large-scale video datasets for classiﬁcation [24], [25], object detection [20], [23] and multiple object tracking [26], [27], [28] have been proposed.”
StrongSORT: Make DeepSORT Great Again (2022, methodology): “Camera movements exist in multiple benchmarks [31, 44, 19].”
Towards Real-Time Multi-Object Tracking (2019, methodology): “The dominant strategy to this problem, i.e., the trackingby-detection paradigm (Milan et al. 2016; Yu et al. 2016; Choi 2015), breaks MOT down to two steps: 1) the detection step, in which targets in single video frames are localized; and 2) the association step, where detected targets are assigned…”
TAP-Vid: A Benchmark for Tracking Any Point in a Video (2022, neutral): “Though box and segment tracking has been widely benchmarked [11, 24, 47, 49, 77, 79, 82, 84], we focus here on methods that track points.”
DiffusionDet: Diffusion Model for Object Detection (2022, background): “As a fundamental visual recognition task, it has become the cornerstone of many related recognition scenarios, such as instance segmentation [36, 53], pose estimation [9, 22], action recognition [32, 81], object tracking [46,65], and visual relationship detection [45,62].”
Associating Objects with Transformers for Video Object Segmentation (2021, background): “As to related areas of VOS, the simple yet effective identification mechanism should also be promising for many tasks requiring multi-object matching, such as interactive video object segmentation [14, 17], video instance segmentation [1, 26, 31], and multi-object tracking [15, 27, 30].”
DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion (2021, background): “Dataset MOT17 [22] MOT20 [9] DanceTrack”
Quasi-Dense Similarity Learning for Multiple Object Tracking (2020, methodology): “We conduct experiments not only on the MOT [28] benchmark but also on the other brand-new large-scale benchmarks including BDD100K [51], Waymo [41], and TAO [8].”
HOTA: A Higher Order Metric for Evaluating Multi-object Tracking (2020, methodology): “We evaluate 37 diﬀerent trackers [11,76,7,64,51,84,89,33, 16,82,30,47,15,35,42,85,26,29,17,34,46,41,70,25,31,2, 62,63,9,10,3,69,4,22,37,5] on MOT17 [54].”
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark (2020, positive): “In multi-object tracking, the MOT challenge [64] provides 21 videos.”
Online Multi-object Tracking Using CNN-Based Single Object Tracker with Spatial-Temporal Attention Mechanism (2017, methodology): “We compare our algorithm, denoted by STAM, with several state-of-the-art MOT tracking methods on the test sequences of MOT15 and MOT16 benchmarks.”
Joint Object Detection and Multi-Object Tracking with Graph Neural Networks (2021, methodology): “Moreover, when comparing with prior joint MOT methods without modeling object relations such as [22], [30], our method achieves better performance as shown in 2DMOT15/MOT16/MOT17 benchmarks, suggesting that adding object relation modeling is useful in joint MOT.”