STMTrack: Template-free Visual Tracking with Space-time Memory Networks

By Zhihong Fu, Qingjie Liu, Zehua Fu, Yunhong Wang. Computer Vision and Pattern Recognition, 2021

337 citations recorded by the source corpus.

Boosting performance of the offline trained siamese trackers is getting harder nowadays since the fixed information of the template cropped from the first frame has been almost thoroughly mined, but they are poorly capable of resisting target appearance changes. Existing trackers with template updating mechanisms rely on time-consuming numerical optimization and complex hand-designed strategies to achieve competitive performance, hindering them from real-time tracking and practical applications. In this paper, we propose a novel tracking framework built on top of a space-time memory network that is competent to make full use of historical information related to the target for better adapting to appearance variations during tracking. Specifically, a novel memory mechanism is introduced, which stores the historical information of the target to guide the tracker to focus on the most informative regions in the current frame. Furthermore, the pixel-level similarity computation of the memory network enables our tracker to generate much more accurate bounding boxes of the target. Extensive experiments and comparisons with many competitive trackers on challenging large-scale benchmarks, OTB-2015, TrackingNet, GOT-10k, LaSOT, UAV123, and VOT2018, show that, without bells and whistles, our tracker outperforms all previous state-of-the-art real-time methods while running at 37 FPS. The code is available at https: //github.com/fzh0917/STMTrack.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Genetics

View Paper: https://www.semanticscholar.org/paper/811ffb185bc90ac5d02d6dbfbcdb6173756b52ef

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Genetics

Citations in the corpus

Learning Target Candidate Association to Keep Track of What Not to Track (2021, background): “Keep Keep Super Pr STM Siam Siam Track Track CRACT TrDiMP TransT DiMP DiMP Track AttN R-CNN KYS DiMP Fast [22] [43] [6] [11] [16] [23] [50] [42] [4] [3]”
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding (2023, background): “There are some prior works exploring memory models [59] in various vision tasks in videos, such as video object segmentation (VOS) [18,30,55,56], multi-object tracking (MOT) [2,14,29,73], visual object tracking (VOT) [38, 42, 77, 84], and action understanding [68].”
Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework (2022, methodology): “SiamFC [1] RT-MDNet [21] ECO [8] Ocean [50] ATOM [7] DiMP50 [2] STMTrack [15] TransT [5] STARK [47] OSTrack -246 OSTrack -384”
ODTrack: Online Dense Temporal Token Learning for Visual Tracking (2024, methodology): “STMTrack(Fu et al. 2021) and TrDiMP(Wang et al. 2021a) employ attention mechanism to effectively extract contextual information along the temporal dimension.”
AiATrack: Attention in Attention for Transformer Visual Tracking (2022, methodology): “The update procedure is more efficient than the previous practices [19,53,58], as we directly reuse the encoded features.”
SwinTrack: A Simple and Strong Baseline for Transformer Tracking (2021, background): “[7] [41] [18] are the very first works that introduce the Transformer architecture to the visual object tracking.”