Deformable Siamese Attention Networks for Visual Object Tracking

By Yu Yu, Yilei Xiong, Weilin Huang, Matthew R. Scott. Computer Vision and Pattern Recognition, 2020

369 citations recorded by the source corpus.

Siamese-based trackers have achieved excellent performance on visual object tracking. However, the target template is not updated online, and the features of target template and search image are computed independently in a Siamese architecture. In this paper, we propose Deformable Siamese Attention Networks, referred to as SiamAttn, by introducing a new Siamese attention mechanism that computes deformable self-attention and cross-attention. The self-attention learns strong context information via spatial attention, and selectively emphasizes interdependent channel-wise features with channel attention. The crossattention is capable of aggregating rich contextual interdependencies between the target template and the search image, providing an implicit manner to adaptively update the target template. In addition, we design a region refinement module that computes depth-wise cross correlations between the attentional features for more accurate tracking. We conduct experiments on six benchmarks, where our method achieves new state-of-the-art results, outperforming recent strong baseline, SiamRPN++, by 0.464 to 0.537 and 0.415 to 0.470 EAO on VOT 2016 and 2018.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5eca15a355b2a9a1e80879e850afe49d3c398c53

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking (2021, background): “Similarly, non-local neural network [55] also introduces a self-attention block to acquire global representations, which has been adopted in many vision tasks including visual object tracking [59].”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “5GHz CPU, RTX 2080Ti GPU Python PyTorch 70 BB CSA [191] ResNet-50 GOT-10k DAF Intel I9 CPU, 64GB RAM, RTX 2080Ti GPU Python PyTorch 100 CM SiamAttn [197] ResNet-50 COCO, YouTube-VOS, LaSOT, TrackingNet DAF N/A, RTX 2080Ti GPU Python PyTorch 33 BB, SGM SiamBAN [198] ResNet-50 ILSVRC-VID, YTBB, COCO, ”
Transformer Tracking (2021, methodology): “SiamAttn [46] explores both self-attention and cross branch attention to improve the discriminative ability of target features before applying depth-wise cross correlation.”
Changer: Feature Interaction is What You Need for Change Detection (2022, background): “For example, [44] uses channel-wise cross-attention to learn mutual information from dual branches for object tracking.”
Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking (2022, background): “Siamese network is a representative paradigm in visual object tracking [2, 25, 24, 9, 45, 43], which usually consists of a Siamese backbone for feature extraction, an interactive head (e.”
AiATrack: Attention in Attention for Transformer Visual Tracking (2022, background): “Some efforts [62,21,19] have been made to introduce the attention structure to visual tracking.”
Learning Target Candidate Association to Keep Track of What Not to Track (2021, background): “Keep Keep Super Pr STM Siam Siam Track Track CRACT TrDiMP TransT DiMP DiMP Track AttN R-CNN KYS DiMP Fast [22] [43] [6] [11] [16] [23] [50] [42] [4] [3]”
SwinTrack: A Simple and Strong Baseline for Transformer Tracking (2021, background): “Later, numerous extensions have been presented to improve Siamese tracking, including deeper backbone [19], multi-stage architecture [10, 11], anchor-free Siamese trackers [41], deformable attention [37], to name a few.”