Transferring Rich Feature Hierarchies for Robust Visual Tracking

By Naiyan Wang, Siyi Li, Abhinav Kumar Gupta, Dit-Yan Yeung. 2015

316 citations recorded by the source corpus.

Convolutional neural network (CNN) models have demonstrated great success in various computer vision tasks including image classification and object detection. However, some equally important tasks such as visual tracking remain relatively unexplored. We believe that a major hurdle that hinders the application of CNN to visual tracking is the lack of properly labeled training data. While existing applications that liberate the power of CNN often need an enormous amount of training data in the order of millions, visual tracking applications typically have only one labeled example in the first frame of each video. We address this research issue here by pre-training a CNN offline and then transferring the rich feature hierarchies learned to online tracking. The CNN is also fine-tuned during online tracking to adapt to the appearance of the tracked target specified in the first video frame. To fit the characteristics of object tracking, we first pre-train the CNN to recognize what is an object, and then propose to generate a probability map instead of producing a simple class label. Using two challenging open benchmarks for performance evaluation, our proposed tracker has demonstrated substantial improvement over other state-of-the-art trackers.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Open-World Object Proposals without Learning to Classify (2021, background): “They have become the integral pre-processing steps for many computer vision systems, including object detection [33,41,42,45,59,74], segmentation [3,8,13], object discovery [12, 16, 47], weakly supervised object detection [5, 21, 53], visual tracking [35, 63], content-aware retargeting [51], etc.”
Object-Proposal Evaluation Protocol is ‘Gameable’ (2015, background): “Object proposal algorithms have quickly become the defacto pre-processing step in a number of vision pipelines – object detection [12–21], segmentation [22–26], object discovery [27–30], weakly supervised learning of object-object interactions [31, 32], content aware media retargeting [33], action r”
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation (2015, methodology): “This transfer learning strategy is widely adopted for object detection [8, 13, 32, 27], semantic segmentation [12, 19, 11, 5, 4, 20], visual tracking [29], and other visual recognition tasks.”
Empirical Evaluation of Rectified Activations in Convolutional Network (2015, background): “Convolutional neural network (CNN) has made great success in various computer vision tasks, such as image classiﬁcation (Krizhevsky et al., 2012; Szegedy et al., 2014), object detection(Girshick et al., 2014) and tracking(Wang et al., 2015).”
Learning Multi-domain Convolutional Neural Networks for Visual Tracking (2015, background): “Despite such huge success of CNNs, only a limited number of tracking algorithms using the representations from CNNs have been proposed so far [10, 20, 28, 39].”
Fully-Convolutional Siamese Networks for Object Tracking (2016, methodology): “SO-DLT [7] and MDNet [9] both train a convolutional network for a similar detection task in an offline phase, then at test-time use SGD to learn a detector with examples extracted from the video itself as in the conventional tracking-as-detector-learning paradigm.”
UvA-DARE (Digital Academic Repository) Siamese Instance Search for Tracking (2016, methodology): “[47] pre-trains a convolutional neural network for measuring generic objectness on ImageNet 2014 detection set and adapts the network online to predict the target-specific objectness.”
BING: Binarized normed gradients for objectness estimation at 300fps (2014, unknown): “A generic objectness measure has great potential to be used as a pre-filter for many vision tasks, including object detection [3–5], visual tracking [6, 7], object discovery [8, 9], semantic segmentation [10, 11], content aware image retargeting [12], and action recognition [13].”
Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning (2017, methodology): “Recently, CNN-based methods [36, 21, 22, 34, 14, 35, 31, 24, 12] have been proposed to learn the tracking models.”
The Visual Object Tracking VOT2015 Challenge Results (2015, background): “The reader is referred to [71] for more details.”
The Visual Object Tracking VOT2016 Challenge Results (2016, background): “The reader is referred to [49] for more details.”
Siamese Instance Search for Tracking (2016, methodology): “[47] pre-trains a convolutional neural network for measuring generic objectness on ImageNet 2014 detection set and adapts the network online to predict the target-specific objectness.”
Understanding and Diagnosing Visual Tracking Systems (2015, background): “Since subjective bias [24] in the results can be caused by selection of videos, this practice makes it infeasible to give a fair comparison of different trackers.”
End-to-End Representation Learning for Correlation Filter Based Tracking (2017, background): “[21, 7, 9, 31]), which have shown that CNNs and CFs are complementary and their combination results in improved performance.”
Factors in Finetuning Deep Model for Object Detection with Long-Tail Distribution (2016, unknown): “detection”