Spatial-Temporal Relation Networks for Multi-Object Tracking

By Jiarui Xu, Zheng Zhang, Han Hu. 2019

263 citations recorded by the source corpus.

Recent progress in multiple object tracking (MOT) has shown that a robust similarity score is a key to the success of trackers. A good similarity score is expected to reflect multiple cues, e.g. appearance, location, and topology, over a long period of time. However, these cues are heterogeneous, making them hard to be combined in a unified network. As a result, existing methods usually encode them in separate networks or require a complex training approach. In this paper, we present a unified framework for similarity measurement based on spatial-temporal relation network which could simultaneously encode various cues and perform reasoning across both spatial and temporal domains. We also study the feature representation of a tracklet-object pair in depth, showing a proper design of the pair features can well empower the trackers. The resulting approach is named spatial-temporal relation networks (STRN). It runs in a feed-forward way and can be trained in an end-to-end manner. The state-of-the-art accuracy was achieved on all of the MOT15$\sim$17 benchmarks using public detection and online settings.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Occluded Video Instance Segmentation: A Benchmark (2021, background): “[84,52] exploit the topology between objects to track the occluded objects.”
Track to Detect and Segment: An Online Multi-Object Tracker (2021, methodology): “MOT was dominated by the tracking-by-detection ( TBD ) paradigm over the past years [58, 6, 66, 52, 33, 5, 38, 48, 54].”
HOTA: A Higher Order Metric for Evaluating Multi-object Tracking (2020, methodology): “We evaluate 37 different trackers [11,76,7,64,51,84,89,33, 16,82,30,47,15,35,42,85,26,29,17,34,46,41,70,25,31,2, 62,63,9,10,3,69,4,22,37,5] on MOT17 [54].”
FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020, methodology): “Somemethods (Sadeghian et al. 2017;Xu et al. 2019; Shan et al. 2020) propose to fuse multiple cues (i.e. motion, appearance and location) to get more reliable similarity.”
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking (2020, background): “ CRF TRACK [Xiang et al., 2020] Appearance, geometry (L) 3 CRF re-id 7 7 7 Tracktor [Bergmann et al., 2019] Re-id (L) 3 | MC, re-id 7 3 3 KCF [Chu et al., 2019] Re-id (L) 3 Multicut re-id 3 3 3 STRN [Xu et al., 2019] Geometry, appearance (L) 3 Hungarian algorithm | 7 7 3 JBNOT [Henschel et al., 2019”
Rethinking the Competition Between Detection and ReID in Multiobject Tracking (2020, background): “Moreover, some works [17], [18] introduce graph neural networks to further improve matching capacity.”
Memory Enhanced Global-Local Aggregation for Video Object Detection (2020, methodology): “Like our method that tries to aggregate information beyond a small local range, [29, 20, 28, 33] share a similar spirit and shows superior results in different areas.”
SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation (2021, unknown): “Others learn tracking representations by exploiting the appearance similarity [33, 42, 35] or re-identify vanished objects [17, 36, 37].”
GSM: Graph Similarity Model for Multi-Object Tracking (2020, background): “[ Xu et al. , 2019 ] utilized relation network to encode the relations information, which makes the individual object differentiable.”
Segment as Points for Efficient Online Multi-Object Tracking and Segmentation (2020, background): “Previous works [31, 36, 8] have demonstrated that features concerning position, appearance, scale, shape, and nearby objects, are useful for tracking.”
Disentangled Non-Local Neural Networks (2020, unknown): “Since then, such modeling methods have demonstrated great e↵ectiveness in many vision tasks, such as image classification [20], object detection [19, 13], semantic segmentation [35], video object detection [9, 33, 14, 6] and tracking [34], and action recognition [31].”