ECO: Efficient Convolution Operators for Tracking

By Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg. Computer Vision and Pattern Recognition, 2016

2,488 citations recorded by the source corpus.

In recent years, Discriminative Correlation Filter (DCF) based methods have significantly advanced the state-of-the-art in tracking. However, in the pursuit of ever increasing tracking performance, their characteristic speed and real-time capability have gradually faded. Further, the increasingly complex models, with massive number of trainable parameters, have introduced the risk of severe over-fitting. In this work, we tackle the key causes behind the problems of computational complexity and over-fitting, with the aim of simultaneously improving both speed and performance. We revisit the core DCF formulation and introduce: (i) a factorized convolution operator, which drastically reduces the number of parameters in the model, (ii) a compact generative model of the training sample distribution, that significantly reduces memory and time complexity, while providing better diversity of samples, (iii) a conservative model update strategy with improved robustness and reduced complexity. We perform comprehensive experiments on four benchmarks: VOT2016, UAV123, OTB-2015, and TempleColor. When using expensive deep features, our tracker provides a 20-fold speedup and achieves a 13.0% relative gain in Expected Average Overlap compared to the top ranked method [12] in the VOT2016 challenge. Moreover, our fast variant, using hand-crafted features, operates at 60 Hz on a single CPU, while obtaining 65.0% AUC on OTB-2015.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a87cc499cf101b3697cacc65094b4b6590e0d061

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Target Candidate Association to Keep Track of What Not to Track (2021, background): “Many online trackers [12, 3, 13] employ a memory to store previous predictions to fine-tune the tracker.”
Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking (2021, methodology): “Discriminative correlation filter (DCF) tackles the visual tracking by solving the ridge regression in Fourier domain, which exhibits attractive efficiency [20, 35, 34, 15, 37, 53, 11, 8].”
Visual Tracking via Adaptive Spatially-Regularized Correlation Filters (2019, methodology): “Usually, the CF-based trackers [18, 12, 11, 15, 8] exploit large numbers of cyclically shifted samples for learning, and convert the correlation operations in the spatial domain to the element-wise multiplications in the frequency domain, thereby reducing the computation complexity and improving…”
Transformer Tracking (2021, methodology): “Moreover, some popular online trackers (e.g., ECO [8], ATOM [9], and DiMP [2]) also heavily rely on the correlation operation.”
Target-Aware Deep Tracking (2019, methodology): “Deep features from multiple layers have also integrated for visual tracking [10, 32, 7, 3].”
Low-Light Image and Video Enhancement Using Deep Learning: A Survey (2021, methodology): “We then use a tracker [92] to track the object in consecutive frames of the”
Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark (2021, background): “We also find that Siamese network based trackers usually achieve better results than other trackers like multi-domain based trackers [26, 44, 45], regression based trackers [21], and correlation filter based trackers [1,8,22].”
Energy-Based Models for Deep Probabilistic Regression (2020, background): “Dataset Metric ECO SiamFC MDNet UPDT DaSiamRPN SiamRPN++ ATOM ATOM∗ Ours [9] [1] [38] [2] [64] [31] [8]”
Learning Attentions: Residual Attentional Siamese Network for High Performance Online Visual Tracking (2018, methodology): “Recently, deep learning models [8,36,43,49], have become an essential oracle to improve the tracking accuracy, mainly due to their large model capacities and strong feature learning abilities.”
SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking (2019, methodology): “52 Titan X Python ECO [25] 0.”
Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Object Tracking (2018, methodology): “ECO [59] applies a projection matrix to realise channel compression.”
Robust Anchor Embedding for Unsupervised Video Person re-IDentification in the Wild (2018, background): “Accordingly, image frames within each sequence are assumed to belong to the same person identity, which could be ensured with effective tracking algorithms [7].”
Learning Spatio-Temporal Transformer for Visual Tracking (2021, neutral): “SiamFC [2] RT-MDNet [23] ECO [10] Ocean [69] LightTrack [60] SiamRPN++ [28] ATOM [11] DiMP50 [3] TransT [6] STARK-S50 STARK-ST50 STARK-ST101”
Siam R-CNN: Visual Tracking by Re-Detection (2019, methodology): “Other recent developments in VOT include using domain specific layers with online learning [50], learning an adaptive spatial filter regularizer [14], exploiting categoryspecific semantic information [61], using continuous [17] or factorized [15] convolutions, and achieving accurate bounding box pre”