DeepTrack: Learning Discriminative Feature Representations by Convolutional Neural Networks for Visual Tracking

By Hanxi Li, Yi Li, Fatih Murat Porikli. 2014

153 citations recorded by the source corpus.

Defining hand-crafted feature representations needs expert knowledge, requires timeconsuming manual adjustments, and besides, it is arguably one of the limiting factors of object tracking. In this paper, we propose a novel solution to automatically relearn the most useful feature representations during the tracking process in order to accurately adapt appearance changes, pose and scale variations while preventing from drift and tracking failures. We employ a candidate pool of multiple Convolutional Neural Networks (CNNs) as a data-driven model of different instances of the target object. Individually, each CNN maintains a specific set of kernels that favourably discriminate object patches from their surrounding background using all available low-level cues. These kernels are updated in an online manner at each frame after being trained with just one instance at the initialization of the corresponding CNN. Given a frame, the most promising CNNs in the pool are selected to evaluate the hypothesises for the target object. The hypothesis with the highest score is assigned as the current detection window and the selected models are retrained using a warm-start back-propagation which optimizes a structural loss function. In addition to the model-free tracker, we introduce a class-specific version of the proposed method that is tailored for tracking of a particular object class such as human faces. Our experiments on a large selection of videos from the recent benchmarks demonstrate that our method outperforms the existing state-of-the-art algorithms and rarely loses the track of the target object.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent advances in convolutional neural networks (2015, background): “Instead of learning one complicated and powerful CNN model for all the appearance observations in the past, [144] uses a relatively small number of filters in the CNN within a framework equipped with a temporal adaptation mechanism.”
Learning Multi-domain Convolutional Neural Networks for Visual Tracking (2015, methodology): “Despite such huge success of CNNs, only a limited number of tracking algorithms using the representations from CNNs have been proposed so far [10, 20, 28, 39].”
Transferring Rich Feature Hierarchies for Robust Visual Tracking (2015, background): “Some other approaches to the incorporation of deep learning in tracking include using an ensemble [39] and maintaining a pool of CNNs [22].”
CREST: Convolutional Residual Learning for Visual Tracking (2017, methodology): “They include stacked denoising autoencoder [44], incremental learning [26], SVM classification [19] and fully connected neural network [33].”
Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network (2015, methodology): “On the other hand, (Li et al., 2014) proposes a target-speciﬁc CNN for object tracking, where the CNN is trained incrementally during tracking with new examples obtained online.”
Meta-Tracker: Fast and Robust Online Adaptation for Visual Object Trackers (2018, background): “Recently, deep learning features have begun to play an important role in correlation ﬁlters [46, 12, 9, 34, 50, 39, 30].”
FusionSeg: Learning to Combine Motion and Appearance for Fully Automatic Segmentation of Generic Objects in Videos (2017, methodology): “Deep learning with motion Deep learning for combining motion and appearance in videos has proven to be useful in several other computer vision tasks such as video classification [32, 22], action recognition [43, 20], object tracking [25, 52, 29] and even computation of optical flow [8].”
Siamese Instance Search for Tracking (2016, background): “[25] learns a target classifier online, which is fundamentally hampered by a lack of data.”
Modeling and Propagating CNNs in a Tree Structure for Visual Tracking (2016, background): “For example, the performance of [22], which is based on a shallow custom neural network, is not as successful as recent tracking algorithms based on shallow feature learning.”
Convolutional Features for Correlation Filter Based Visual Tracking (2015, background): “[27] propose a deep tracking framework using a candidate pool of multiple CNNs.”
Learning to Track at 100 FPS with Deep Regression Networks (2016, background): “Second, most trackers evaluate a finite number of samples and choose the highest scoring one as the tracking output [26,27,37,30,39,24,33].”
UvA-DARE (Digital Academic Repository) Siamese Instance Search for Tracking (2016, background): “[25] learns a target classifier online, which is fundamentally hampered by a lack of data.”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “SVM [54], DPST [55], CCOT [56], GOTURN [57], SiamFC [58], SINT [59], MDNet [60], HDT [61], STCT [62], RPNT [63], DeepTrack [64], [65], CNT [66], CF-CNN [67], TCNN [68], RDLT [69], PTAV [70], [71], CREST [72], UCT/UCTLite [73], DSiam/DSiamM [74], TSN [75], WECO [76], RFL [77], IBCCF [78], DTO [79]], ”
Hierarchical Convolutional Features for Visual Tracking (2015, methodology): “Existing deep learning based trackers [30, 21, 29, 18] typically draw positive and negative training samples around the estimated target location to incrementally learn a classifier over features extracted from a CNN.”
Unveiling the Power of Deep Tracking (2018, background): “Li et al. [11] tackle the tracking problem in an end-to-end fashion by training a classiﬁer online.”