Distractor-aware Siamese Networks for Visual Object Tracking

By Zheng Zhu, Qiang Wang, Bo Li, Wei Wu, Junjie Yan, Weiming Hu. European Conference on Computer Vision, 2018

1,303 citations recorded by the source corpus.

Recently, Siamese networks have drawn great attention in visual tracking community because of their balanced accuracy and speed. However, features used in most Siamese tracking approaches can only discriminate foreground from the non-semantic backgrounds. The semantic backgrounds are always considered as distractors, which hinders the robustness of Siamese trackers. In this paper, we focus on learning distractor-aware Siamese networks for accurate and long-term tracking. To this end, features used in traditional Siamese trackers are analyzed at first. We observe that the imbalanced distribution of training data makes the learned features less discriminative. During the off-line training phase, an effective sampling strategy is introduced to control this distribution and make the model focus on the semantic distractors. During inference, a novel distractor-aware module is designed to perform incremental learning, which can effectively transfer the general embedding to the current video domain. In addition, we extend the proposed approach for long-term tracking by introducing a simple yet effective local-to-global search region strategy. Extensive experiments on benchmarks show that our approach significantly outperforms the state-of-the-arts, yielding 9.6% relative gain in VOT2016 dataset and 35.9% relative gain in UAV20L dataset. The proposed tracker can perform at 160 FPS on short-term benchmarks and 110 FPS on long-term benchmarks.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/776bc8955e801f6965e85b35d8e2dd6f2f1498ad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Safety, Risk, Reliability and Quality

Citations in the corpus

Learning the Model Update for Siamese Trackers (2019, methodology): “While the original SiamFC tracker [1] does not perform any model update, more recent Siamese trackers [1, 41, 27, 50] have adopted (1) to update their templates.”
Deformable Siamese Attention Networks for Visual Object Tracking (2020, background): “[44] extended the SiamRPN by developing distractor-aware training.”
Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Object Tracking (2018, neutral): “ECO∗ CFCF∗ [74] CFWCR∗ [78] LSART∗ [79] UPDT∗ [80] SiamRPN∗ [81] MFT∗ [18] LADCF ∗”
Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking (2018, methodology): “By formulating object tracking as a matching problem, Siamese trackers [2, 17, 19, 23, 45, 46, 51, 59] aim to learn offline a generic similarity function from a large set of videos.”
Target-Aware Deep Tracking (2019, methodology): “The proposed algorithm is compared with the state-of-the-art trackers, including the correlation ﬁlters based trackers, such as SRDCF [9], Staple [1], MCPF [48], CCOT [10], ECO [7], BACF [17], DRT [38], STRCF [22], and MCCT-H [43]; the non-real-time deep trackers such as MDNet [29], CREST [35], LSAR”
Attention-Guided Unified Network for Panoptic Segmentation (2018, neutral): “Attention-based Modules: Attention-based modules have been widely applied in visual tasks, including image processing, video understanding, and object tracking [7,19,37, 46, 47].”
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark (2020, background): “Owing to balanced efficiency and accuracy, deep Siamese tracking has been extended by many later works [27, 36, 49, 51, 78, 80, 95].”
Siamese Box Adaptive Network for Visual Tracking (2020, methodology): “We compare our tracker with other 9 state-of-art real-time trackers, including DiMP [2], ATOM [5], SiamRPN++ [20], DaSiamRPN [52], SiamRPN [21], ECO [6], ECO-HC [6], SRDCF [7], SAMF [24].”
Learning Target Candidate Association to Keep Track of What Not to Track (2021, methodology): “As one of the most challenging factors, co-occurrence of distractor objects similar in appearance to the target is a common problem in real-world tracking applications [4, 56, 46].”
Video Instance Segmentation Tracking With a Modified VAE Architecture (2020, methodology): “Detection-free tracking methods [5, 58, 37, 62, 36, 69] track objects given a manual initialization of a ﬁxed number of objects in the ﬁrst frame.”
Semantics Disentangling for Text-To-Image Generation (2019, methodology): “Inspired by the advantages of Siamese structure used in different tasks [32, 33, 4, 10, 43] which can find the similarity between a pair of sequences, we treat our discriminator as an image comparator so as to preserve the semantic consistency among the generated images as long as their descriptions”
Learning Discriminative Model Prediction for Tracking (2019, background): “DRT RCO UPDT DaSiam- MFT LADCF ATOM SiamRPN++ DiMP-18 DiMP-50 [37] [20] [3] RPN [45] [20] [43] [6] [21]”
Probabilistic Regression for Visual Tracking (2020, neutral): “RCO UPDT DaSiam- MFT LADCF ATOM SiamRPN++ DiMP18 DiMP50 PrDiMP18 PrDiMP50 [25] [4] RPN [51] [25] [48] [9] [28] [3] [3]”
Deeper and Wider Siamese Networks for Real-Time Visual Tracking (2019, methodology): “Previous methods [2, 12, 13, 20, 40, 42] commonly utilize the classic and relatively shallow AlexNet [18] as the backbone network φ in this framework.”
Graph Attention Tracking (2020, positive): “Based on SiamRPN [16], DaSiamRPN [37] designs a distractor-aware module to perform incremental learning and obtains much more discriminative features against semantic distractors.”
CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning (2019, methodology): “For task 3, we also experiment with a state-of-the-art visual tracking method (Zhu et al., 2018).”