Deep Affinity Network for Multiple Object Tracking

By Shijie Sun, Naveed Akhtar, Huansheng Song, Ajmal Saeed Mian, Mubarak Shah. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

365 citations recorded by the source corpus.

Multiple Object Tracking (MOT) plays an important role in solving many fundamental problems in video analysis and computer vision. Most MOT methods employ two steps: Object Detection and Data Association. The first step detects objects of interest in every frame of a video, and the second establishes correspondence between the detected objects in different frames to obtain their tracks. Object detection has made tremendous progress in the last few years due to deep learning. However, data association for tracking still relies on hand crafted constraints such as appearance, motion, spatial proximity, grouping etc. to compute affinities between the objects in different frames. In this paper, we harness the power of deep learning for data association in tracking by jointly modeling object appearances and their affinities between different frames in an end-to-end fashion. The proposed Deep Affinity Network (DAN) learns compact, yet comprehensive features of pre-detected objects at several levels of abstraction, and performs exhaustive pairing permutations of those features in any two frames to infer object affinities. DAN also accounts for multiple objects appearing and disappearing between video frames. We exploit the resulting efficient affinity computations to associate objects in the current frame deep into the previous frames for reliable on-line tracking. Our technique is evaluated on popular multiple object tracking challenges MOT15, MOT17 and UA-DETRAC. Comprehensive benchmarking under twelve evaluation metrics demonstrates that our approach is among the best performing techniques on the leader board for these challenges. The open source implementation of our work is available at https://github.com/shijieS/SST.git.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9f69382b0112b5467b8118eafaade5d30720c3b2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yi Yang
2. Liang Zheng
3. Luc Van Gool
4. Tao A. Xiang
5. Dacheng Tao
6. Chen Change Loy
7. Mubarak Shah
8. Qi Tian

Citations in the corpus

• Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking (2020, background): “[8] combined the detection and feature extraction module in a network, [9,10] combined the feature extraction and data association module.”
• Advances in adversarial attacks and defenses in computer vision: A survey (2021, methodology): “We note that the original tracker used by [27] follows tracking-by-detection paradigm [203].”
• Track to Detect and Segment: An Online Multi-Object Tracker (2021, neutral): “7% 19,701 152,622 6,033 3,147 57 DAN[37] TPAMI 2019 52.”
• Towards Real-Time Multi-Object Tracking (2019, background): “Ones that make efforts to model the association process by an end-to-end neural network (Sun et al. 2019; Zhu et al. 2018).”
• FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020, neutral): “9 MOT17 SST [69] 52.”