End-to-End Representation Learning for Correlation Filter Based Tracking

By Jack Valmadre, Luca Bertinetto, João F. Henriques, Andrea Vedaldi, Philip H. S. Torr. 2017

1,460 citations recorded by the source corpus.

The Correlation Filter is an algorithm that trains a linear template to discriminate between images and their translations. It is well suited to object tracking because its formulation in the Fourier domain provides a fast solution, enabling the detector to be re-trained once per frame. Previous works that use the Correlation Filter, however, have adopted features that were either manually designed or trained for a different task. This work is the first to overcome this limitation by interpreting the Correlation Filter learner, which has a closed-form solution, as a differentiable layer in a deep neural network. This enables learning deep features that are tightly coupled to the Correlation Filter. Experiments illustrate that our method has the important practical benefit of allowing lightweight architectures to achieve state-of-the-art performance at high framerates.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Meta-learning with differentiable closed-form solvers (2018, methodology): “In similar spirit, Valmadre et al. (2017) proposed a method to backpropagate through the solution of a closed-form problem.”
Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects (2018, neutral): “Single-object tracking commonly use Siamese networks, which can be seen as an RNN unrolled over two time-steps (Valmadre et al., 2017).”
Real-Time MDNet (2018, methodology): “[21] regress response maps between target objects and input images to maximize the score at the ground-truth target location.”
The Sixth Visual Object Tracking VOT2018 Challenge Results (2018, methodology): “Instead of using the pretrained network in [27], the presented tracker uses the network introduced in [81] to extract deep features.”
Learning the Model Update for Siamese Trackers (2019, methodology): “We bor-row a linear update rate from CFNet [40] as γ = 0 .”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “strategies to reduce the risk of over-fitting [74], [83], [86], [96], [104], [145], [187]–[189], [216], ii) background suppression [74], [104], iii) formulating tracking task as a one-shot local detection task [104], [116], iv) and giving higher weights to important feature channels or score maps [8”
High Performance Visual Tracking with Siamese Region Proposal Network (2018, methodology): “Recently, Siamese networks have drawn great attention in visual tracking community because of their balanced accuracy and speed [13, 12, 4, 35, 36] .”
MixFormer: End-to-End Tracking with Iterative Mixed Attention (2022, positive): “To gain benefits from end-to-end training, CFNet [65] integrated the correlation filter into deep network.”
Meta-Tracker: Fast and Robust Online Adaptation for Visual Object Trackers (2018, background): “Recently, deep learning features have begun to play an important role in correlation ﬁlters [46, 12, 9, 34, 50, 39, 30].”
Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things (2020, methodology): “Later, different deep learning methods have been proposed to improve the classical methods by learning multi-resolution deep features [72], end-to-end representation learning [73], and leveraging siamese networks [74].”
Unveiling the Power of Deep Tracking (2018, background): “The work of [15] integrate DCF into the Siamese framework [19].”
Learning Dynamic Memory Networks for Object Tracking (2018, methodology): “To address this, [30] proposes model updating using linear interpolation of new templates with a small learning rate, but does only sees modest improvements in accuracy.”
Dynamic Kernel Distillation for Efficient Pose Estimation in Videos (2019, methodology): “It can also fast distill pose kernels in a one-shot manner, avoiding complex iterating utilized by previous online kernel learning models [4, 27].”