Need for Speed: A Benchmark for Higher Frame Rate Object Tracking

By Hamed Kiani Galoogahi, Ashton Fagg, Chen Huang, Deva Ramanan, Simon Lucey. IEEE International Conference on Computer Vision, 2017

506 citations recorded by the source corpus.

In this paper, we propose the first higher frame rate video dataset (called Need for Speed - NfS) and benchmark for visual object tracking. The dataset consists of 100 videos (380K frames) captured with now commonly available higher frame rate (240 FPS) cameras from real world scenarios. All frames are annotated with axis aligned bounding boxes and all sequences are manually labelled with nine visual attributes - such as occlusion, fast motion, background clutter, etc. Our benchmark provides an extensive evaluation of many recent and state-of-the-art trackers on higher frame rate sequences. We ranked each of these trackers according to their tracking accuracy and real-time performance. One of our surprising conclusions is that at higher frame rates, simple trackers such as correlation filters outperform complex methods based on deep networks. This suggests that for practical applications (such as in robotics or embedded vision), one needs to carefully tradeoff bandwidth constraints associated with higher frame rate acquisition, computational costs of real-time analysis, and the required application accuracy. Our dataset and benchmark allows for the first time (to our knowledge) systematic exploration of such issues, and will be made available to allow for further research in this space.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/703505a00579c0aa67712836acc41d94fa6d6edc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

How to Train Your Energy-Based Model for Regression (2020, unknown): “Results are also reported on the 30 FPS version of the need for speed (NFS) [24] dataset, containing 100 videos with fast motions.”
Know Your Surroundings: Exploiting Scene Information for Object Tracking (2020, methodology): “PDT [4] tracker achieves the best results with an AUC score of 70:4. Our approach obtains results comparable with the state-of-the-art, while outperforming the baseline DiMP-50 by over 1% in AUC. NFS [14]: The need for speed dataset consists of 100 challenging videos captured using a high frame rate”
SeqTrack: Sequence to Sequence Learning for Visual Object Tracking (2023, methodology): “TNL2K, NFS and UAV123.”
Learning Spatio-Temporal Transformer for Visual Tracking (2021, methodology): “Table 5: Success score (%) comparisons on the collected large-scale benchmark NOTU and its subsets [24, 58, 33, 42].”
Transforming Model Prediction for Tracking (2022, methodology): “Comparison with the state of the art on the OTB-100 [61], NFS [23] and UAV123 [49] datasets in terms of AUC score.”
Energy-Based Models for Deep Probabilistic Regression (2020, background): “Hyperparameters are set on the OTB [56] and NFS [25] datasets, containing 100 videos each.”
AiATrack: Attention in Attention for Transformer Visual Tracking (2022, methodology): “Need for Speed (NfS) [30] is a dataset that contains 100 videos with fast-moving objects.”
Learning Discriminative Model Prediction for Tracking (2019, methodology): “Experiments are performed on a combined dataset containing the entire OTB-100 [37], NFS (30 FPS version) [9] and UAV123 [24] datasets.”
The Seventh Visual Object Tracking VOT2019 Challenge Results (2019, background): “[28] introduced a high-frame-rate dataset to analyze trade-offs between tracker speed and robustness.”
Vision Meets Drones: A Challenge (2018, unknown): “5k 2016 UAV123 [9] drone 123 110k 2016 NfS [25] life 100 383k 2017 POT 210 [26] planar objects 210 105.”
Probabilistic Regression for Visual Tracking (2020, methodology): “Following [3], we perform this analysis of our approach on the combined OTB-100 [46], UAV123 [32] and NFS [14] dataset, totaling 323 diverse videos, and report the average over 5 runs.”
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking (2018, background): “NfS [14] provides 100 sequences with a high framerate of 240 fps, aiming to analyze the effects of appearance variations on tracking performance.”
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark (2020, background): “Currently, popular dense tracking benchmarks include OTB [83, 84], TC128 [57], VOT [45], NUS-PRO [47], UAV [65], NfS [30], CDTB [60] and GOT-10k [41].”
Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework (2022, methodology): “NFS, UAV123 and TNL2K.”
Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark (2021, unknown): “Datasets #Videos #Min #Mean #Max #Total #FR #Attributes Aim Absent Lang-A Lang-I SAV Adv DA OTB50 [57] 51 71 578 3,872 29K 30 fps 11 Eval OTB100 [58] 100 71 590 3,872 59K 30 fps 11 Eval TC-128 [37] 128 71 429 3,872 55K 30 fps 11 Eval VOT-2017 [28] 60 41 356 1,500 21K 30 fps Eval NUS-PRO [29] 365 146”
Autoregressive Visual Tracking (2023, methodology): “Main Results We evaluate the performance of our proposed ARTrack256, ARTrack384 and ARTrack-L384 on several benchmarks, including GOT-10k [31], TrackingNet [47], LaSOT [19], LaSOText [18], TNL2K [57], UAV123 [46] and NFS [33].”