Siam R-CNN: Visual Tracking by Re-Detection

By Paul Voigtlaender, Jonathon Luiten, Philip H. S. Torr, Bastian Leibe. Computer Vision and Pattern Recognition, 2019

661 citations recorded by the source corpus.

We present Siam R-CNN, a Siamese re-detection architecture which unleashes the full power of two-stage object detection approaches for visual object tracking. We combine this with a novel tracklet-based dynamic programming algorithm, which takes advantage of re-detections of both the first-frame template and previous-frame predictions, to model the full history of both the object to be tracked and potential distractor objects. This enables our approach to make better tracking decisions, as well as to re-detect tracked objects after long occlusion. Finally, we propose a novel hard example mining strategy to improve Siam R-CNN's robustness to similar looking objects. Siam R-CNN achieves the current best performance on ten tracking benchmarks, with especially strong results for long-term tracking. We make our code and models available at www.vision.rwth-aachen.de/page/siamrcnn.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/069ccdbab6ea6ca2d9c3b75c76360ca1e4e9a5e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Target Candidate Association to Keep Track of What Not to Track (2021, methodology): “Keep Keep Super Pr STM Siam Siam Track Track CRACT TrDiMP TransT DiMP DiMP Track AttN R-CNN KYS DiMP Fast [22] [43] [6] [11] [16] [23] [50] [42] [4] [3]”
Transformer Tracking (2021, methodology): “This table shows that the proposed method obtains the best performance, better than other trackers by a significant margin except SiamRCNN [39], but SiamR-CNN merely runs less than 5fps in our machine, while our tracker runs at 50 fps.”
Learning Spatio-Temporal Transformer for Visual Tracking (2021, methodology): “The proposed tracker achieves state-of-the-art performance on multiple challenging short-term and long-term benchmarks, while running at real-time speed, being 6× faster than Siam R-CNN [54].”
End-to-end Recurrent Multi-Object Tracking and Trajectory Prediction with Relational Reasoning (2019, methodology): “acker is evaluated on all future frames. The performance is typically measured asintersection-over-union (IOU)averaged across all frames in which the object is present. On many VOT datasets, SIAMRCNN [28] is currentlystate-of-the-art (SOTA). In each frame, it employs a pre-trainedregion proposal net”
Autoregressive Visual Tracking (2023, background): “Current prevailing trackers [2, 10, 15, 23, 36, 37, 54, 69] typically rely on the matching between the template and search images.”
STMTrack: Template-free Visual Tracking with Space-time Memory Networks (2021, background): “From the perspective of existing works [46, 37], experiences, and intuitions, target information from the first frame and the previous frame plays an important role for the target localization in the current frame.”
Ocean: Object-aware Anchor-free Tracking (2020, methodology): “CRPN ECO STRCF LADCF ATOM SRCNN SiamRPN++ DiMP Ocean Ocean [9] [5] [23] [45] [4] [39] [21] [2] offline online EAO ↑ 0.”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “00GHz, 24GB RAM, GTX Titan X GPU Python TensorFlow 43 CM, OS SiamRCNN [200] Faster R-CNN, ResNet-101-FPN ILSVRC-VID, COCO, YouTube-VOS, GOT-10k, LaSOT DAF N/A, Tesla V100 GPU Python TensorFlow 4.”
Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark (2021, background): “In addition to learn powerful feature representation and conduct a local search for tracking, some trackers attempt to achieve robust tracking by global search [18, 24, 51, 52, 55, 61].”
How to Train Your Energy-Based Model for Regression (2020, neutral): “Compared to previous approaches, only the very recent SiamRCNN [51] achieves results competitive with our DiMP-NCE+.”
MixFormer: End-to-End Tracking with Iterative Mixed Attention (2022, methodology): “Generally, most trackers [3,7,12,28,47] used ResNet as the backbone.”