Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network

By Seunghoon Hong, Tackgeun You, Suha Kwak, Bohyung Han. 2015

852 citations recorded by the source corpus.

We propose an online visual tracking algorithm by learning discriminative saliency map using Convolutional Neural Network (CNN). Given a CNN pre-trained on a large-scale image repository in offline, our algorithm takes outputs from hidden layers of the network as feature descriptors since they show excellent representation performance in various general visual recognition problems. The features are used to learn discriminative target appearance models using an online Support Vector Machine (SVM). In addition, we construct target-specific saliency map by backprojecting CNN features with guidance of the SVM, and obtain the final tracking result in each frame based on the appearance model generatively constructed with the saliency map. Since the saliency map reveals spatial configuration of target effectively, it improves target localization accuracy and enables us to achieve pixel-level target segmentation. We verify the effectiveness of our tracking algorithm through extensive experiment on a challenging benchmark, where our method illustrates outstanding performance compared to the state-of-the-art tracking algorithms.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent advances in convolutional neural networks (2015, methodology): “[146] propose a visual tracking algorithm based on a pre-trained CNN, where the network is trained”
Learning Multi-domain Convolutional Neural Networks for Visual Tracking (2015, methodology): “A few recent approaches [39, 20] transfer CNNs pretrained on a large-scale dataset constructed for image classiﬁcation, but the representation may not be very effective due to the fundamental difference between classiﬁcation and tracking tasks.”
Siamese Instance Search for Tracking (2016, positive): “[17] focuses on learning target-specific saliency map using pre-trained ImageNet network.”
Learning Uncertain Convolutional Features for Accurate Saliency Detection (2017, methodology): “As a preprocessing procedure in computer vision, saliency detection has greatly benefited many practical applications such as object retargeting [6, 40], scene classification [37], semantic segmentation [34] and visual tracking [29, 14].”
Detect Globally, Refine Locally: A Novel Approach to Saliency Detection (2018, background): “It has been shown effective in a wide range of applications including person identification [2], visual tracking [9], image captioning [7, 8], robot navigation [6] and visual question answering [21].”
Online Multi-Expert Learning for Visual Tracking (2020, neutral): “Apart from the trackers used on the OTB2013 benchmark, we add another state-of-art trackers including CNN-SVM [63], LCT [64] and TGPR [65].”
Online Model Distillation for Efficient Video Inference (2018, methodology): “Tracking: Traditional object tracking methods [20, 12, 15] and more recent methods built upon deep feature hierarchies [27, 45, 17, 31] can be viewed as a form of rapid online learning of appearance models from video.”
Learning to Track at 100 FPS with Deep Regression Networks (2016, background): “Some researchers have also attempted to use neural networks for tracking within the traditional online training framework [26,27,34,37,35,30,39,7,24,16], showing state-of-the-art results [30,7,21].”
Salient Object Detection With Pyramid Attention and Salient Edges (2019, background): “It can be widely applied for improving a variety of vision tasks, such as object proposal generation [2], object segmentation [42, 44], photo cropping [45, 41], and video object tracking [13], among others.”
RGB-D salient object detection: A survey (2020, background): “SOD plays a key role in a range of real-world applications, such as stereo matching [11], image understanding [12], co-saliency detection [13], action recognition [14], video detection and segmentation [15]–[18], semantic segmentation [19], [20], medical image segmentation [21]–[23], object tracking”
Visual Tracking with Fully Convolutional Networks (2015, methodology): “Prior works [6, 35, 12] attempted to transfer offline learned DNN features (e.”
Learning Dynamic Siamese Network for Visual Object Tracking (2017, background): “Recently, such classification & updating tracking scheme has been developed via online fine-tuned deep networks, thus has achieved much better (or the best) tracking accuracy [18] either by transferring some pre-trained networks [14, 25, 11] for specific tracking tasks [28, 21, 17] or by directly le”
Hedged Deep Tracking (2016, methodology): “Comparison with state-of-the-art trackers We compare our algorithm with 8 recent state-of-theart trackers: DLT [30], CNN-SVM [18], KCF [17], MEEM [36], Struck [14], CXT [8], TLD [20], and SCM [41].”
Staple: Complementary Learners for Real-Time Tracking (2015, methodology): “Moreover, our method also outperforms recent trackers published after the benchmark such as MEEM [41], DSST [8], TGPR [13], EBT [38] and also trackers that make use of deep conv-nets like CNN-SVM [19] and SO-DLT [37], while running at a signiﬁ-cantly higher frame rate.”
Understanding and Diagnosing Visual Tracking Systems (2015, methodology): “Although there exist even more powerful features such as those extracted by the convolutional neural network (CNN) and they indeed can yield state-of-the-art performance [34, 16], naı̈ve application of this approach will incur high computational cost which is highly undesirable for tracking applicat”
Understanding Deep Learning Techniques for Image Segmentation (2019, background): “Surveillance system can also be extended to various applications such as object tracking [82], searching [3], anomaly detection [173], threat detection [137], traffic control [208] and so on.”
R³Net: Recurrent Residual Refinement Network for Saliency Detection (2018, background): “Inferring salient objects benefits many applications, such as weakly-supervised object detection [Baisheng Lai, 2017] and visual tracking [Hong et al., 2015].”
Modeling and Propagating CNNs in a Tree Structure for Visual Tracking (2016, methodology): “The proposed algorithm is compared with the eight state-of-the-art trackers including HCT [26], CNN-SVM [14], FCNT [34], SRDCF [7], MUSTer [15], MEEM [38], DSST [5], and Struck [12].”