ByteTrack: Multi-Object Tracking by Associating Every Detection Box

By Yifu Zhang, Pei Sun, Yi Jiang, Dongdong Yu, Zehuan Yuan, Ping Luo, Wenyu Liu, Xinggang Wang. European Conference on Computer Vision, 2021

2,498 citations recorded by the source corpus.

Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects in videos. Most methods obtain identities by associating detection boxes whose scores are higher than a threshold. The objects with low detection scores, e.g. occluded objects, are simply thrown away, which brings non-negligible true object missing and fragmented trajectories. To solve this problem, we present a simple, effective and generic association method, tracking by associating almost every detection box instead of only the high score ones. For the low score detection boxes, we utilize their similarities with tracklets to recover true objects and filter out the background detections. When applied to 9 different state-of-the-art trackers, our method achieves consistent improvement on IDF1 score ranging from 1 to 10 points. To put forwards the state-of-the-art performance of MOT, we design a simple and strong tracker, named ByteTrack. For the first time, we achieve 80.3 MOTA, 77.3 IDF1 and 63.1 HOTA on the test set of MOT17 with 30 FPS running speed on a single V100 GPU. ByteTrack also achieves state-of-the-art performance on MOT20, HiEve and BDD100K tracking benchmarks. The source code, pre-trained models with deploy versions and tutorials of applying to other trackers are released at https://github.com/ifzhang/ByteTrack.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7d1ff4ac2390759cbe60dd46b2b9bcabd4a90db4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors (2022, background): “For example, multi-object tracking [90, 91], autonomous driving [17, 39], robotics [34, 55], medical image analysis [33, 44], etc.”
AiATrack: Attention in Attention for Transformer Visual Tracking (2022, methodology): “We believe that the proposed AiA module could also be beneficial in other related tasks where the Transformer architecture can be applied to perform feature aggregation and information propagation, such as video object segmentation [60,33,15,39], video object detection [23] and multi-object tracking”
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (2024, background): “…image segmentation [9, 8, 26, 29, 78, 88, 79, 25, 79, 16, 28], referring object detection and segmentation [41, 37, 86], object tracking [67, 84], image generation [75, 54, 48, 45, 23, 14, 52, 22, 82, 44], image editing [42, 1, 2, 53, 21], human-centric perception and understanding [72, 71,…”
BoT-SORT: Robust Associations Multi-Pedestrian Tracking (2022, methodology): “Thus, tracking-by-detection trackers mainly focus on improving data association, while exploiting deep learning trends [58, 12].”
Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking (2022, methodology): “We thank Yifu Zhang for providing the code of ByteTrack.”
MOTR: End-to-End Multiple-Object Tracking with TRansformer (2021, methodology): “Experimental results on DanceTrack show that MOTR significantly outperforms state-of-the-art method, ByteTrack [42] by 6.”
DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion (2021, methodology): “Lastly, BYTE [38] uses a high-tolerance strategy to select detection results into the association stage.”
DETRs Beat YOLOs on Real-time Object Detection (2023, background): “Real-time object detection is a significant research field and has a wide range of applications, such as object tracking [39, 42], video surveillance [24], autonomous driving [2, 38], etc.”
DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving (2024, positive): “End-to-end autonomous driving has gained signiﬁcant attention in recent years due to advancements in perception models (detection [5, 17, 24, 42], tracking [54–56], online mapping [ 27, 28, 32], etc .)”
StrongSORT: Make DeepSORT Great Again (2022, methodology): “To be speciﬁc, we ﬁrst equip DeepSORT with a strong detector [18] following [63] and embedding model [30].”