Online Multi-object Tracking Using CNN-Based Single Object Tracker with Spatial-Temporal Attention Mechanism

By Qi Chu, Wanli Ouyang, Hongsheng Li, Xiaogang Wang, Nenghai Yu. 2017

372 citations recorded by the source corpus.

In this paper, we propose a CNN-based framework for online MOT. This framework utilizes the merits of single object trackers in adapting appearance models and searching for target in the next frame. Simply applying single object tracker for MOT will encounter the problem in computational efficiency and drifted results caused by occlusion. Our framework achieves computational efficiency by sharing features and using ROI-Pooling to obtain individual features for each target. Some online learned target-specific CNN layers are used for adapting the appearance model for each target. In the framework, we introduce spatial-temporal attention mechanism (STAM) to handle the drift caused by occlusion and interaction among targets. The visibility map of the target is learned and used for inferring the spatial attention map. The spatial attention map is then applied to weight the features. Besides, the occlusion status can be estimated from the visibility map, which controls the online updating process via weighted loss on training samples with different occlusion statuses in different frames. It can be considered as temporal attention mechanism. The proposed algorithm achieves 34.3% and 46.0% in MOTA on challenging MOT15 and MOT16 benchmark dataset respectively.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Occluded Video Instance Segmentation: A Benchmark (2021, methodology): “In Multi-Object Tracking, [12,94] utilize the attention module to attend to the visible parts of objects.”
Spatial-Temporal Relation Networks for Multi-Object Tracking (2019, methodology): “and the hand-crafted location cues are integrated with appearance cue in recent works [46, 12, 63].”
TrackFormer: Multi-Object Tracking with Transformers (2021, methodology): “In this work, we introduce the tracking-by-attention paradigm which not only applies attention for data association [11, 67] but jointly performs tracking and detection.”
Real-Time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification (2018, methodology): “[8] used a binary classifier and single object tracker for online multi-object tracking.”
Multiple object tracking: A literature review (2014, background): “The formulation and modeling of the target association problem using deep neural networks [201, 202, 203, 204] need more research efforts, although the first attempt to employ sequential neural networks for online MOT has been made very recently.”
Deep Affinity Network for Multiple Object Tracking (2018, neutral): “6 AM [89] online 30.”
Deep learning in video multi-object tracking: A survey (2019, methodology): “In [124], a model of stacked CNNs was proposed.”
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking (2020, methodology): “…past years all leverage the representational power of deep learning for learning association afﬁnities, learning to adapt appearance models online (Chu et al. 2019, 2017; Kim et al. 2018; Zhu et al. 2018) and learning to regress tracking targets (Bergmann et al. 2019; Chu et al. 2019, 2017; Zhu…”
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild (2018, methodology): “In real life, generic object tracking has a wide range of applications in diverse fields [44] such as in surveillance [36], augmented reality [42], biology [43] and robotics [41].”
Learning to Restore Low-Light Images via Decomposition-and-Enhancement (2020, background): “, object detection [31] and tracking [8]), or image editing tasks (e.”
Online Multi-Object Tracking with Dual Matching Attention Networks (2018, methodology): “Recently, several on-line approaches [10,27,36,42,2] using deep learning models have been proposed.”
Learning Attentions: Residual Attentional Siamese Network for High Performance Online Visual Tracking (2018, background): “They have spread to other areas such as image classification [24, 26, 48], pose estimation [13], multi-object tracking [6], etc.”
Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking (2020, background): “STAM [12] 46.”