Understanding and Diagnosing Visual Tracking Systems

By Naiyan Wang, Jianping Shi, Dit-Yan Yeung, Jiaya Jia. IEEE International Conference on Computer Vision, 2015

364 citations recorded by the source corpus.

Several benchmark datasets for visual tracking research have been created in recent years. Despite their usefulness, whether they are sufficient for understanding and diagnosing the strengths and weaknesses of different trackers remains questionable. To address this issue, we propose a framework by breaking a tracker down into five constituent parts, namely, motion model, feature extractor, observation model, model updater, and ensemble post-processor. We then conduct ablative experiments on each component to study how it affects the overall result. Surprisingly, our findings are discrepant with some common beliefs in the visual tracking research community. We find that the feature extractor plays the most important role in a tracker. On the other hand, although the observation model is the focus of many studies, we find that it often brings no significant improvement. Moreover, the motion model and model updater contain many details that could affect the result. Also, the ensemble post-processor can improve the result substantially when the constituent trackers have high diversity. Based on our findings, we put together some very elementary building blocks to give a basic tracker which is competitive in performance to the state-of-the-art trackers. We believe our framework can provide a solid baseline when conducting controlled experiments for visual tracking research.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/bf97833a354d7faa3793599ac5c1ec1eb3eb7e07

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Siamese Instance Search for Tracking (2016, methodology): “As a preliminary demonstration that SINT can be further improved by employing more advanced online components, we also evaluate a variant of SINT, coined SINT+, which uses an adaptive candidate sampling strategy suggested by [48] and optical flow [4].”
Learning Dynamic Siamese Network for Visual Object Tracking (2017, background): “The first one adopts the classical classification & updating pipeline, which maintains an online updated classifier [29, 13] or object appearance model [34] to optimally select the most probable candidate sample as the tracked object at a coming frame [18, 30].”
UvA-DARE (Digital Academic Repository) Siamese Instance Search for Tracking (2016, methodology): “As a preliminary demonstration that SINT can be further improved by employing more advanced online components, we also evaluate a variant of SINT, coined SINT+, which uses an adaptive candidate sampling strategy suggested by [48] and optical flow [4].”
Kernel Cross-Correlator (2017, background): “(Wang et al. 2016) formulates convolutional neural network as ensemble learning and shows its effectiveness in visual tracking.”
Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Object Tracking (2018, background): “It has been demonstrated that robust feature representation plays the most essential role in high-performance visual object tracking [72], [73].”
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild (2018, background): “[64] have further shown its potential in actively mining the training samples in unlabeled videos, paving the way for the more automatic learning system.”
The Visual Object Tracking VOT2015 Challenge Results (2015, methodology): “Four complementary trackers were chosen to be used in ensemble, namely, DAT [58], DSST [11], Baseline [73] and ASMS [70].”
Multi-cue Correlation Filters for Robust Visual Tracking (2018, methodology): “To enhance the tracking robustness and obtain reliable results, the ensemble approach treats the trackers as black boxes and takes only the bounding boxes returned by them as input [43].”
Learning to Track at 100 FPS with Deep Regression Networks (2016, methodology): “during test time) [15,3,36,19], with no offline training being performed.”
Visual Object Tracking for Unmanned Aerial Vehicles: A Benchmark and New Motion Models (2017, methodology): “Some exploit the representation power of CNNs by pre-training them on an external large-scale image dataset (Hong et al. 2015; Wang et al. 2015a) while others (Nam and Han 2016) utilize existing tracking videos to capture more domain-specific information.”