SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks

By Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, Junjie Yan. 2018

2,218 citations recorded by the source corpus.

Siamese network based trackers formulate tracking as convolutional feature cross-correlation between target template and searching region. However, Siamese trackers still have accuracy gap compared with state-of-the-art algorithms and they cannot take advantage of feature from deep networks, such as ResNet-50 or deeper. In this work we prove the core reason comes from the lack of strict translation invariance. By comprehensive theoretical analysis and experimental validations, we break this restriction through a simple yet effective spatial aware sampling strategy and successfully train a ResNet-driven Siamese tracker with significant performance gain. Moreover, we propose a new model architecture to perform depth-wise and layer-wise aggregations, which not only further improves the accuracy but also reduces the model size. We conduct extensive ablation studies to demonstrate the effectiveness of the proposed tracker, which obtains currently the best results on four large tracking benchmarks, including OTB2015, VOT2018, UAV123, and LaSOT. Our model will be released to facilitate further studies based on this problem.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, methodology): “This operation is analogous to the search function in visual tracking literature [35, 36, 37] where ot[Tpick] is the exemplar and ot is the search area, or in retrieval systems literature where ot[Tpick] is the query and ot is the key.”
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark (2020, background): “, MDNet [68], TRACA [9], CFNet [76], SiamFC [6], StructSiam [92], DSiam [33], SINT [75], ATOM [15], DiMP [7], VITAL [74], SiamRPN++ [48], DaSiamRPN [95], SiamDW [93], C-RPN [27] and SiamMask [82], GlobalTrack [42]), correlation trackers with hand-crafted features (e.”
Siamese Box Adaptive Network for Visual Tracking (2020, methodology): “Although anchor-based trackers [21, 39, 20] can handle changes in scale and aspect ratio, it is necessary to carefully design and fix the parameters of the anchor boxes.”
Deformable Siamese Attention Networks for Visual Object Tracking (2020, methodology): “As observed in [24], each channel map of the high-level convolutional features usually responses for a specific object class.”
Graph Attention Tracking (2020, neutral): “Existing correlation based information embedding meth-ods [2, 16, 15] take the whole target feature as a unity to match with the search features.”
NestFuse: An Infrared and Visible Image Fusion Architecture Based on Nest Connection and Spatial/Channel Attention Models (2020, background): “The experimental results and analyses show that the proposed fusion framework demonstrates state-of-the-art fusion performance.”
Siam R-CNN: Visual Tracking by Re-Detection (2019, methodology): “In the past, the problem of distractors has mainly been approached by strong spatial priors from previous predictions [4, 38, 37], or by online adaptation [1, 21, 2, 56, 23, 57, 32].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, methodology): “[206] B. Li, W. Wu, Q. Wang, F. Zhang, J. Xing, and J. Yan, ‘‘SiamRPN++: Evolution of Siamese visual tracking with very deep networks,’’ in Proc.”
Visual Commonsense R-CNN (2020, background): “Today’s computer vision systems are good at telling us “what” ( e.g ., classiﬁcation [23, 31], segmentation [22, 39]) and “where” ( e.g ., detection [54, 38], tracking [30, 34]), yet bad at knowing “why”, e.g ., why is it dog ?”
Few-Shot Object Detection With Attention-RPN and Multi-Relation Detector (2019, methodology): “Here the support features X is used as the kernel to slide on the query feature map [58, 59] in a depth-wise cross correlation manner [60].”
SwinTrack: A Simple and Strong Baseline for Transformer Tracking (2021, methodology): “SiamRPN++ [25] 35 54 TrSiam [41] 35 TrDiMP [41] 26 TransT [7] 50 STARK-ST50 [44] 42 24 STARK-ST101 [44] 32 42 SwinTrack-T 98 23 SwinTrack-B 52 91 SwinTrack-B-384 45 91”
Probabilistic Regression for Visual Tracking (2020, methodology): “A comparison of our approach with state-of-the-art trackers DiMP [3] and SiamRPN++ [28].”
Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection (2022, background): “Owing to their evident appearance discrepancy, it is challenging to fuse visually appealing images and/or to support higher-level vision tasks such as segmentation [4,29], tracking [2,7], and detection [32], by making full use of the complementary information from the infrared and visible.”
ODTrack: Online Dense Temporal Token Learning for Visual Tracking (2024, methodology): “As shown in Fig.1(a), these offline meth-ods(Bertinetto et al. 2016; Li et al. 2019; Chen et al. 2021; (b) Our online video-level tracking method based on video sequence sampling and temporal token propagation.”