Online Multi-Object Tracking with Dual Matching Attention Networks

By Ji Zhu, Hua Yang, Nian Liu, Minyoung Kim, Wenjun Zhang, Ming-Hsuan Yang. 2018

372 citations recorded by the source corpus.

In this paper, we propose an online Multi-Object Tracking (MOT) approach which integrates the merits of single object tracking and data association methods in a unified framework to handle noisy detections and frequent interactions between targets. Specifically, for applying single object tracking in MOT, we introduce a cost-sensitive tracking loss based on the state-of-the-art visual tracker, which encourages the model to focus on hard negative distractors during online learning. For data association, we propose Dual Matching Attention Networks (DMAN) with both spatial and temporal attention mechanisms. The spatial attention module generates dual attention maps which enable the network to focus on the matching patterns of the input image pair, while the temporal attention module adaptively allocates different levels of attention to different samples in the tracklet to suppress noisy observations. Experimental results on the MOT benchmark datasets show that the proposed algorithm performs favorably against both online and offline trackers in terms of identity-preserving metrics.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Occluded Video Instance Segmentation: A Benchmark (2021, methodology): “In Multi-Object Tracking, [12,94] utilize the attention module to attend to the visible parts of objects.”
Tracking Objects as Points (2020, background): “Most modern trackers [2, 7, 23, 33, 35, 41, 47, 50, 58] follow the tracking-by-detection paradigm.”
SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation (2021, background): “Recent multi-object tracking mainly follows the tracking-by-detection paradigm [9, 18, 34, 42, 48], which first detects objects and then predicts the temporal association of all detections.”
ByteTrack: Multi-Object Tracking by Associating Every Detection Box (2021, methodology): “A large number of methods [72,15,3,13,90,9,29] focus on improving the tracking performance based on these given detection results.”
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking (2020, methodology): “, 2018] DeepMatching, geometric 3 Frank-Wolfe algorithm Head detector 7 7 7 DMAN [Zhu et al., 2018] Appearance (L), geometry 3 — — 3 3 3 eHAF [Zhu et al.”
Learning a Neural Solver for Multiple Object Tracking (2019, background): “ed to some extent. In the second group, several works adopt a simpler and easier to optimize graph structure, and focus instead on improving edge cost deﬁnition by leveraging deep learning techniques [39,61,59,75,67]. By exploiting siamese convolutional neural networks (CNN), these approaches can en”
TransTrack: Multiple-Object Tracking with Transformer (2020, background): “Most state-of-the-art multi-object trackers [43, 39, 7, 42, 47, 41, 4] follow the tracking-by-detection paradigm.”
Multiple object tracking: A literature review (2014, methodology): “The modules like attention mechanism [205], LSTM [114, 97] are also employed by researchers for solving the MOT problem.”
GSM: Graph Similarity Model for Multi-Object Tracking (2020, methodology): “Recently, deep networks have been adopted to MOT for modeling appearance of objects [Zhu et al. , 2018].”
FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020, background): “Some works such as Xiang et al. (2015), Zhu et al. (2018), Chu and Ling (2019), Chu et al. (2019) leverage sophisticated single object tracking methods to get accurate object locations and reduce false negatives.”
Deep learning in video multi-object tracking: A survey (2019, methodology): “Many algorithms in fact have chosen to train networks using either actual detections [96] or by manually adding noise and errors to the ground truth trajectories [93, 115], although this may slow the training procedure sometimes and not always be feasible [60].”
Track to Detect and Segment: An Online Multi-Object Tracker (2021, methodology): “MOT was dominated by the tracking-by-detection ( TBD ) paradigm over the past years [58, 6, 66, 52, 33, 5, 38, 48, 54].”
Tracking Without Bells and Whistles (2019, methodology): “[67] use single object tracking trained networks to create tracklets for further postprocessing into trajectories.”
Spatial-Temporal Relation Networks for Multi-Object Tracking (2019, methodology): “and the hand-crafted location cues are integrated with appearance cue in recent works [46, 12, 63].”
TrackFormer: Multi-Object Tracking with Transformers (2021, methodology): “In this work, we introduce the tracking-by-attention paradigm which not only applies attention for data association [11, 67] but jointly performs tracking and detection.”
Joint Object Detection and Multi-Object Tracking with Graph Neural Networks (2021, methodology): “Online methods [10], [14], [44], [45] only require information up to the current frame for data association and can be useful to online applications.”