Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking

By Feng Li, Cheng Tian, Wangmeng Zuo, Lei Zhang, Ming-Hsuan Yang. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

785 citations recorded by the source corpus.

Discriminative Correlation Filters (DCF) are efficient in visual tracking but suffer from unwanted boundary effects. Spatially Regularized DCF (SRDCF) has been suggested to resolve this issue by enforcing spatial penalty on DCF coefficients, which, inevitably, improves the tracking performance at the price of increasing complexity. To tackle online updating, SRDCF formulates its model on multiple training images, further adding difficulties in improving efficiency. In this work, by introducing temporal regularization to SRDCF with single sample, we present our spatial-temporal regularized correlation filters (STRCF). The STRCF formulation can not only serve as a reasonable approximation to SRDCF with multiple training samples, but also provide a more robust appearance model than SRDCF in the case of large appearance variations. Besides, it can be efficiently solved via the alternating direction method of multipliers (ADMM). By incorporating both temporal and spatial regularization, our STRCF can handle boundary effects without much loss in efficiency and achieve superior performance over SRDCF in terms of accuracy and speed. Compared with SRDCF, STRCF with hand-crafted features provides a 5Ã— speedup and achieves a gain of 5.4% and 3.6% AUC score on OTB-2015 and Temple-Color, respectively. Moreover, STRCF with deep features also performs favorably against state-of-the-art trackers and achieves an AUC score of 68.3% on OTB-2015.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9f45b55af027503fab557f55f70e81e43c6c1db7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Know Your Surroundings: Exploiting Scene Information for Object Tracking (2020, methodology): “A popular method to learn the target appearance model is the discriminative correlation ﬁlters (DCF) [5,21,10,9,26,33].”
A systematic review on overfitting control in shallow and deep neural networks (2021, methodology): “To control the complexity of a model, there are different methods, see e.g., (Chen and Yao 2009; Li et al. 2018; Zhu et al. 2018; Liu et al. 2014; Eigenmann and Nossek 1999; Finnoff et al. 1993).”
AutoTrack: Towards High-Performance Visual Tracking for UAV With Automatic Spatio-Temporal Regularization (2020, methodology): “δ is a constant to adjust the weight of local response variations, and u is inherited from STRCF [14] to mitigate boundary effects.”
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking (2018, background): “…deep features ( e.g ., HCFT [37] and ECO [7]) and regularization techniques ( e.g ., BACF [15], SRDCF [10], CSRDCF [36], Staple CA [40] and STRCF [29]), ensemble trackers ( e.g ., PTAV [13], LCT [38], MEEM [56] and TLD [24]), sparse trackers ( e.g ., L1APG [2] and ASLA [23]), other…”
ATOM: Accurate Tracking by Overlap Maximization (2018, methodology): “We evaluate the proposed tracker ATOM on five benchmarks: Need for Speed (NFS) [9], UAV123 [24], TrackingNet [25], LaSOT [8], and VOT2018 [17].”
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild (2018, methodology): “works on correlation filters based tracking: CSK [55], KCF [56] and their variants ColorCSK [62], SAMF [58], DSST [48], Staple [45], SRDCF [51], SRDCFdecon [52], CCOT [53], BACF [69], STRCF [81], DeepSTRCF [81], ECO [47], and LDES [82] in our evaluation.”
Fast Online Object Tracking and Segmentation: A Unifying Approach (2018, methodology): “In the semi-supervised setting, VOS methods are initialised with a binary mask [44] and many of them require computa-6 SiamMask SiamMask-2B DaSiamRPN [72] SiamRPN [31] SA Siam R [17] CSRDCF [37] STRCF [32] LSART [56] tionally intensive techniques at test time such as ﬁne-tuning [39, 45, 2, 61], data”
Learning the Model Update for Siamese Trackers (2019, methodology): “We compare with trackers including DRT [37], DeepSTRCF [28], LSART [38], R MCPF [48], SRCT [25], CSRDCF [30], LADCF [44], MFT [22], UPDT [2] and ATOM [8] among others as in [22].”
Visual Tracking via Adaptive Spatially-Regularized Correlation Filters (2019, methodology): “Recently, many researchers have attempted to combine the CF model with deep visual features, making the CF-based trackers achieve state-of-the-art performance [29, 8, 13, 22, 32].”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “The revisited formulation helps the DeepSTRCF to not only endure some appearance variations such as the IV, IPR, OPR, and POC.”
Target-Aware Deep Tracking (2019, methodology): “The proposed algorithm is compared with the state-of-the-art trackers, including the correlation ﬁlters based trackers, such as SRDCF [9], Staple [1], MCPF [48], CCOT [10], ECO [7], BACF [17], DRT [38], STRCF [22], and MCCT-H [43]; the non-real-time deep trackers such as MDNet [29], CREST [35], LSAR”
Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking (2021, background): “Some previous works explore the temporal information using graph neural network [16], spatial-temporal regularization [29], optical flow [64], etc.”
Is First Person Vision Challenging for Object Tracking? (2020, positive): “Specifically, we have considered short-term trackers [46] based on correlation-filters with hand-crafted features (MOSSE [11], DSST [23], KCF [37], Staple [7], BACF [42], DCFNet [79], STRCF [49], MCCTH [78]) and deep features (ECO [21], ATOM [22], DiMP [9], PrDiMP [24], KYS [10]).”
Ocean: Object-aware Anchor-free Tracking (2020, methodology): “CRPN ECO STRCF LADCF ATOM SRCNN SiamRPN++ DiMP Ocean Ocean [9] [5] [23] [45] [4] [39] [21] [2] offline online EAO ↑ 0.”
CoTracker: It is Better to Track Together (2023, background): “We mask out points that have not yet been initialized in the current sliding window, which allows initializing tracks from any point in the video, including in the middle of a window, and to join windows together.”