High-Speed Tracking with Kernelized Correlation Filters

By João F. Henriques, Rui Caseiro, Pedro Martins, Jorge P. Batista. 2014

5,428 citations recorded by the source corpus.

The core component of most modern trackers is a discriminative classifier, tasked with distinguishing between the target and the surrounding environment. To cope with natural image changes, this classifier is typically trained with translated and scaled sample patches. Such sets of samples are riddled with redundancies—any overlapping pixels are constrained to be the same. Based on this simple observation, we propose an analytic model for datasets of thousands of translated patches. By showing that the resulting data matrix is circulant, we can diagonalize it with the discrete Fourier transform, reducing both storage and computation by several orders of magnitude. Interestingly, for linear regression our formulation is equivalent to a correlation filter, used by some of the fastest competitive trackers. For kernel regression, however, we derive a new kernelized correlation filter (KCF), that unlike other kernel algorithms has the exact same complexity as its linear counterpart. Building on it, we also propose a fast multi-channel extension of linear correlation filters, via a linear kernel, which we call dual correlation filter (DCF). Both KCF and DCF outperform top-ranking trackers such as Struck or TLD on a 50 videos benchmark, despite running at hundreds of frames-per-second, and being implemented in a few lines of code (Algorithm 1). To encourage further developments, our tracking framework was made open-source.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles (2016, methodology): “The main advantage of this method is that labeling requires just using a tracking method (they use SURF interest points to detect initial bounding boxes and then tracking via the KCF method [17]).”
Learning to See by Moving (2015, background): “It is believed that biological agents developed perceptual systems for obtaining sensory information about their environment that can aid them in performing actions required for accomplishing their goals [16, 12].”
Unsupervised Learning of Visual Representations Using Videos (2015, methodology): “Given the initial bounding box, we perform tracking using the KCF tracker [19].”
Transitive Invariance for Self-Supervised Visual Representation Learning (2017, methodology): “Note that the KCF tracker does not require any human supervision.”
Learning Multi-domain Convolutional Neural Networks for Visual Tracking (2015, methodology): “[18] formulated kernelized correlation ﬁlters (KCF) using circulant matrices, and efﬁciently incorporated multi-channel features in a Fourier domain.”
Fully-Convolutional Siamese Networks for Object Tracking (2016, methodology): “In addition to the trackers reported by [11], in Figure 3 we also compare against seven more recent state-of-the-art trackers presented in the major computer vision conferences and that can run at frame-rate speed: Staple [33], LCT [34], CCT [35], SCT4 [36], DLSSVM NU [37], DSST [38] and KCFDP [39].”
Object-Centric Representation Learning from Unlabeled Videos (2016, methodology): “After obtaining the initial bounding box (the first patch), they perform tracking using the KCF tracker [35] and track along 30 frames in the video to obtain the second patch.”
Transferring Rich Feature Hierarchies for Robust Visual Tracking (2015, background): “The features learned by a large-scale CNN has demonstrated superior performance in some high-level vision tasks, such as image clas-siﬁcation [18] and object detection [8], as compared to conventional pipelines that use human-engineered features.”
Learning Dynamic Memory Networks for Object Tracking (2018, methodology): “We compare our method MemTrack with 9 recent real-time trackers (≥ 15 fps), including CFNet [30], LMCF [32], ACFN [5], RFL [36], SiamFC [4], SiamFC U [30], Staple [3], DSST [7], and KCF [15] on both OTB-2013 and OTB-2015.”
Matching Pixels Using Co-occurrence Statistics (2018, methodology): “The Lucas-Kanade (LK) image registration algorithm [19] and the Kernelized Correlation Filter (KCF) tracker [11].”
Visual Tracking with Fully Convolutional Networks (2015, methodology): “Success plots of OPE FCNT [0.599] MEEM [0.567] TGPR [0.529] KCF [0.514] SCM [0.499] Struck [0.474] TLD [0.437] DLT [0.436] ASLA [0.434] CXT [0.426] Figure 6.”
Siam R-CNN: Visual Tracking by Re-Detection (2019, background): “These (and many more [7, 27, 46]) only search for the object within a small window of the previous prediction.”
A Novel Performance Evaluation Methodology for Single-Target Trackers (2015, methodology): “The DSST, SAMF and KCF are correlation-filter-based trackers derived from MOSSE [75] that apply holistic models, i.e., a HOG [76].”
Staple: Complementary Learners for Real-Time Tracking (2015, methodology): “In particular, it surpasses trackers like Correlation Filter-based DSST [8], SAMF [29] and KCF [18], colour-based PixelTrack [11], DAT, DATs (with scale) [33] and DGT [6], and also more complex and far slower methods like DMA [40] and SRDCF [9], which operates below 10 FPS.”
Correlation filters with limited boundaries (2014, background): “and their applications [17, 14, 4] are outside the scope of this paper.”
Object Detection in Videos with Tubelet Proposal Networks (2017, background): “We also compare with a state-of-the-art single-object tracking method [10] denoted as “KCF”.”
ECO: Efficient Convolution Operators for Tracking (2016, methodology): “The standard approach in DCF based tracking is to update the model in each frame [4, 9, 20].”