Correlation filters with limited boundaries

By Hamed Kiani Galoogahi, Terence Sim, Simon Lucey. Computer Vision and Pattern Recognition, 2014

342 citations recorded by the source corpus.

Correlation filters take advantage of specific properties in the Fourier domain allowing them to be estimated efficiently: O(N D log D) in the frequency domain, versus O(D3 + N D2) spatially where D is signal length, and N is the number of signals. Recent extensions to correlation filters, such as MOSSE, have reignited interest of their use in the vision community due to their robustness and attractive computational properties. In this paper we demonstrate, however, that this computational efficiency comes at a cost. Specifically, we demonstrate that only 1/D proportion of shifted examples are unaffected by boundary effects which has a dramatic effect on detection/tracking performance. In this paper, we propose a novel approach to correlation filter estimation that: (i) takes advantage of inherent computational redundancies in the frequency domain, (ii) dramatically reduces boundary effects, and (iii) is able to implicitly exploit all possible patches densely extracted from training examples during learning process. Impressive object tracking and detection results are presented in terms of both accuracy and computational efficiency.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a3630dcccab5db268e4a4f1ce22c2e37a26cf5e2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Object Tracking (2018, methodology): “A similar approach was employed in CFLB [12] and BACF [15], forcing the parameters corresponding to background to be exactly zero.”
Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking (2018, methodology): “[17] extend [18] to multiple channels and further speed up the tracker towards real-time.”
Learning Spatially Regularized Correlation Filters for Visual Tracking (2015, methodology): “We provide a comparison of our tracker with 24 state-of-the-art methods from the literature: MIL [2], IVT [28], CT [36], TLD [22], DFT [29], EDFT [12], ASLA [21], L1APG [3], CSK [19], SCM [37], LOT [26], CPF [27], CXT [11], Frag [1], Struck [16], LSHT [17], LSST [32], ACT [10], KCF [20], CFLB [14], ”
Meta-Tracker: Fast and Robust Online Adaptation for Visual Object Trackers (2018, background): “[21] makes it more efficient by taking advantage of circulant matrices, further improved by resolving artificial boundary issues [11, 15].”
Convolutional Features for Correlation Filter Based Visual Tracking (2015, methodology): “Moreover, the filter is optimized directly in the Fourier domain using Gauss-Seidel, while every ADMM iteration in [16] requires a transition between the spatial and Fourier domain.”
VITAL: VIsual Tracking via Adversarial Learning (2018, background): “Extensions include, but are not limited to, kernelized correlation filters [23], scale estimation [10], re-detection [37], spatial regularization [12, 14, 9], ADMM optimization [29], sparse representation [32, 42, 33], CNN feature integrations [36, 43, 64, 28] and end-to-end CNN predictions [57, 55,”
End-to-End Representation Learning for Correlation Filter Based Tracking (2017, background): “Notable efforts have been devoted to its improvement, for example by mitigating the effect of periodic boundaries [10, 15, 8], incorporating multi-resolution feature maps [21, 9] and augmenting the objective with a more robust loss [26].”
Learning Background-Aware Correlation Filters for Visual Tracking (2017, background): “…as a ridge regression objective in the spatial domain: E(h) = 1 2 D X j=1 ||y(j) K X k=1 h > k x k [ ⌧ j ]||2 2 + 2 K X k=1 ||h k ||2 2 (2) where y(j) is the j-th element of y. [ ⌧ j ] is the circular shift operator, and x k [ ⌧ j ] applies a j-step discrete circular shift to the signal x k .”
Learning the Model Update for Siamese Trackers (2019, methodology): “Moreover, they compute the transformation matrix as a closed-form solution on the Fourier domain, which suffers from issues related to the boundary effect [21].”
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking (2016, methodology): “, including scale estimation [9,31], non-linear kernels [23,24], a long-term memory [36], and by alleviating the periodic effects of circular convolution [11,15,18].”
Probabilistic Regression for Visual Tracking (2020, methodology): “The aforementioned confidence-based regression strategy is shared by the previously dominant Discriminative Correlation Filter (DCF) paradigm [5, 10, 12, 15, 22, 31, 39] and the more recent Siamese trackers [2, 17, 28, 29, 44, 50].”
Need for Speed: A Benchmark for Higher Frame Rate Object Tracking (2017, methodology): “For tracking lower frame rate videos, only BACF, Staple, KCF and CFLB can track at or above real time on CPUs.”
Adaptive Decontamination of the Training Set: A Unified Formulation for Discriminative Visual Tracking (2016, methodology): “We perform a comprehensive comparison with 17 recent state-of-the-art trackers: EDFT [7], LSHT [12], DFT [24], ASLA [15], TLD [16], Struck [11], CFLB [8], ACT [6], TGPR [9], KCF [13], DSST [2], SAMF [19], DAT [23], MEEM [29], LCT [22], HCF [21] and SRDCF [5].”
Discriminative Correlation Filter Tracker with Channel and Spatial Reliability (2016, background): “The Galoogahi et al. [24] and Dan-neljan et al. [10] methods both suffer from this problem.”
Kernel Cross-Correlator (2017, methodology): “Fortunately, it is possible to use the existing methods to eliminate this effect (Fernandez et al. 2015; Galoogahi, Sim, and Lucey 2015).”
ECO: Efficient Convolution Operators for Tracking (2016, methodology): “The recent advancement in DCF based tracking performance is driven by the use of multi-dimensional features [13, 15], robust scale estimation [7, 11], non-linear kernels [20], long-term memory components [28], sophisticated learning models [3, 10] and reducing boundary effects [9, 16].”
Multi-cue Correlation Filters for Robust Visual Tracking (2018, methodology): “enhanced by using multi-dimensional features [16, 26, 31], part-based strategies [29, 28], adaptive scale [9, 26], longterm framework [32, 17, 45], end-to-end learning [41, 40] and improved filter training methods [20, 30, 33, 14].”
Fast Online Object Tracking and Segmentation: A Unifying Approach (2018, background): “Performance of Correlation Filter-based trackers has then been notably improved with the adoption of multi-channel formulations [26, 22], spatial constraints [27, 15, 37, 32] and deep features (e.”
Context-Aware Correlation Filter Tracking (2017, neutral): “[4, 6, 10, 11, 14].”