A Twofold Siamese Network for Real-Time Object Tracking

By Anfeng He, Chong Luo, Xinmei Tian, Wenjun Zeng. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

608 citations recorded by the source corpus.

Observing that Semantic features learned in an image classification task and Appearance features learned in a similarity matching task complement each other, we build a twofold Siamese network, named SA-Siam, for real-time object tracking. SA-Siam is composed of a semantic branch and an appearance branch. Each branch is a similaritylearning Siamese network. An important design choice in SA-Siam is to separately train the two branches to keep the heterogeneity of the two types of features. In addition, we propose a channel attention mechanism for the semantic branch. Channel-wise weights are computed according to the channel activations around the target position. While the inherited architecture from SiamFC [3] allows our tracker to operate beyond real-time, the twofold design and the attention mechanism significantly improve the tracking performance. The proposed SA-Siam outperforms all other real-time trackers by a large margin on OTB-2013/50/100 benchmarks.

Subfields: Computer Vision and Pattern Recognition, Global and Planetary Change, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/a3a4471e82260f573d240cc34aeff431cf236571

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Global and Planetary Change

In Safety, Risk, Reliability and Quality

Citations in the corpus

Transformer Tracking (2021, methodology): “For improving the tracking performance, different attention layers (such as channel-wise attention [41, 17], spatial-temporal attention [50], and residual attention [41]) are utilized to enhance the template information within the correlation matching framework.”
Fast Online Object Tracking and Segmentation: A Unifying Approach (2018, neutral): “[4, 31, 71, 67, 18]), we consider the popular SiamFC [4] and SiamRPN [31] as two representative examples.”
Target-Aware Deep Tracking (2019, methodology): “The proposed algorithm is compared with the state-of-the-art trackers, including the correlation ﬁlters based trackers, such as SRDCF [9], Staple [1], MCPF [48], CCOT [10], ECO [7], BACF [17], DRT [38], STRCF [22], and MCCT-H [43]; the non-real-time deep trackers such as MDNet [29], CREST [35], LSAR”
Learning the Model Update for Siamese Trackers (2019, background): “Many trackers have extended the SiamFC architecture [40, 16, 41, 27, 50, 49] for tracking.”
The Seventh Visual Object Tracking VOT2019 Challenge Results (2019, methodology): “He, C. Luo, X. Tian, W. Zeng heanfeng@mail.ustc.edu.cn, cluo@microsoft.com, xinmei@ustc.edu.cn, wezeng@microsoft.com SA-SIAM-R is a variation of the Siamese network-based tracker SA-Siam [35].”
Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking (2018, background): “By formulating object tracking as a matching problem, Siamese trackers [2, 17, 19, 23, 45, 46, 51, 59] aim to learn offline a generic similarity function from a large set of videos.”
Unveiling the Power of Deep Tracking (2018, methodology): “We compare our method with the following state-of-the-art methods: CCOT [7], ECO [1], TCNN [14], VITAL [36], SA-Siam [37], FlowTrack [38], LSART [39], CREST [16], and Staple [3].”
Unsupervised Deep Tracking (2019, methodology): “Some trackers (e.g., SA-Siam [15] and RT-MDNet [19]) adopt pre-trained CNN models (e.g., AlexNet [23] and VGG-M [3]) for network initialization.”
The Sixth Visual Object Tracking VOT2018 Challenge Results (2018, methodology): “cn SA Siam R is a variation of the Siamese network-based tracker SA-Siam [30].”
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild (2018, methodology): “In this work, we consider siamese tracker SiamFC [46] and its variants CFNet [63], DCFNet [75], DSiam [72], SASiam [76], and SiamDW [78]; recurrent tracker Mem-”
Graph Attention Tracking (2020, positive): “Following this similarity-learning work, many researchers try to enhance the Siamese model for feature representation but still leverage the cross-correlation for information embedding [11, 12, 30, 9].”
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark (2020, background): “Owing to balanced efficiency and accuracy, deep Siamese tracking has been extended by many later works [27, 36, 49, 51, 78, 80, 95].”
RGB-T Object Tracking: Benchmark and Baseline (2018, background): “He et al. [41] design a twofold Siamese network that consists of a semantic branch and an appearance branch to improve the discrimination power of SiameFC in tracking.”
Semantics-Enhanced Adversarial Nets for Text-to-Image Synthesis (2019, background): “In other computer vision tasks, such as person search [35], person Re-identification [17, 2], object tracking [11, 45] and Image Captioning [41, 22], visual attention mechanism been studied.”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “task [111], [123], and iv) giving higher weights to important feature channels or score maps [88], [124], [128], [148].”