Know Your Surroundings: Exploiting Scene Information for Object Tracking

By Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte. 2020

396 citations recorded by the source corpus.

Current state-of-the-art trackers only rely on a target appearance model in order to localize the object in each frame. Such approaches are however prone to fail in case of e.g. fast appearance changes or presence of distractor objects, where a target appearance model alone is insufficient for robust tracking. Having the knowledge about the presence and locations of other objects in the surrounding scene can be highly beneficial in such cases. This scene information can be propagated through the sequence and used to, for instance, explicitly avoid distractor objects and eliminate target candidate regions. In this work, we propose a novel tracking architecture which can utilize scene information for tracking. Our tracker represents such information as dense localized state vectors, which can encode, for example, if the local region is target, background, or distractor. These state vectors are propagated through the sequence and combined with the appearance model output to localize the target. Our network is learned to effectively utilize the scene information by directly maximizing tracking performance on video segments. The proposed approach sets a new state-of-the-art on 3 tracking benchmarks, achieving an AO score of 63.6% on the recent GOT-10k dataset.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[22] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte.”
AiATrack: Attention in Attention for Transformer Visual Tracking (2022, methodology): “Several efforts have been made to recalibrate the raw correlations by processing them as features through hand-crafted algorithms [49,5] or learnable blocks [48,43,36,4,47,9].”
Transforming Model Prediction for Tracking (2022, background): “ToMP ToMP Keep STARK STARK Super Pr STM Siam Siam 101 50 Track CRACT ST101 TrDiMP TransT ST50 DiMP DiMP Track AttN R-CNN KYS DiMP [48] [21] [63] [59] [7] [63] [12] [16] [22] [68] [57] [2] [1]”
Learning Target Candidate Association to Keep Track of What Not to Track (2021, methodology): “As one of the most challenging factors, co-occurrence of distractor objects similar in appearance to the target is a common problem in real-world tracking applications [4, 56, 46].”
Is First Person Vision Challenging for Object Tracking? (2020, neutral): “Trackers using an online learning approach [22, 9, 24, 21, 10] generally achieve a below real-time speed.”
Transformer Tracking (2021, methodology): “In this subsection, we compare our TransT method with twelve state-of-the-art trackers published in 2020 (SiamR-CNN [39], Ocean [48], KYS [3], DCFST [49], SiamFC++ [44], PrDiMP [10], CGACD [13], Sia-mAttn [46], MAML [40], D3S [26], SiamCAR [16], and SiamBAN [5]) and six representative trackers prese”
Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark (2021, background): “Other trackers also attain good performance such as LTMU [7], KYS [18], TACT [5], due to the use of joint local and global search scheme.”