Modeling and Propagating CNNs in a Tree Structure for Visual Tracking

By Hyeonseob Nam, Mooyeol Baek, Bohyung Han. arXiv.org, 2016

332 citations recorded by the source corpus.

We present an online visual tracking algorithm by managing multiple target appearance models in a tree structure. The proposed algorithm employs Convolutional Neural Networks (CNNs) to represent target appearances, where multiple CNNs collaborate to estimate target states and determine the desirable paths for online model updates in the tree. By maintaining multiple CNNs in diverse branches of tree structure, it is convenient to deal with multi-modality in target appearances and preserve model reliability through smooth updates along tree paths. Since multiple CNNs share all parameters in convolutional layers, it takes advantage of multiple models with little extra cost by saving memory space and avoiding redundant network evaluations. The final target state is estimated by sampling target candidates around the state in the previous frame and identifying the best sample in terms of a weighted average score from a set of active CNNs. Our algorithm illustrates outstanding performance compared to the state-of-the-art techniques in challenging datasets such as online tracking benchmark and visual object tracking challenge.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0dd92d7a55bbab7c5d12ae3858a6baeb99b65e67

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Real-Time MDNet (2018, methodology): “Recently, Convolutional Neural Networks (CNNs) are very eﬀective in visual tracking [1,2,3,4,5,6,7,8,9,10], but, unfortunately, highly accurate tracking algorithms based on CNNs are often too slow for practical systems.”
Online Multi-Expert Learning for Visual Tracking (2020, neutral): “In [49], Nam et al.”
Multi-cue Correlation Filters for Robust Visual Tracking (2018, methodology): “In Table 4, we list the detailed results of our approaches and the top ranked methods in VOT2016 ( e.g., C-COT [13], TCNN [34] and SSAT).”
Lucid Data Dreaming for Object Tracking (2017, methodology): “Since then, convnets have been used as improved features [11] [32] [58], and eventually to drive the tracking itself [18] [3] [53] [35] [36].”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “To achieve the goal of learning generic representations for target modeling and constructing a more robust target models, the main contributions of methods are classified into: i) offline training of CNNs on large-scale datasets for visual tracking [63], [68], [80], [89], [97], [100], [101], [104], ”
Unveiling the Power of Deep Tracking (2018, methodology): “To ﬁnd the hardest videos in OTB-2015, we consider the per-video results of four deep-feature-based trackers with top overall performance on the dataset: ECO [1], C-COT [7], MDNet [13], and TCNN [14].”
Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking (2018, methodology): “We compare STRCF and DeepSTRCF with state-of-theart trackers, including MDNet [32] (VOT-2015 winner) and TCNN [31] (VOT-2016 winner).”
3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions (2019, background): “Tree-structured Deep Networks: There have been several attempts to represent convolutional neural networks or long short-term memory using tree structures [5, 20, 27, 28, 31].”
Learning Dynamic Memory Networks for Object Tracking (2018, methodology): “Although our MemTrack performs worse than CCOT, TCNN and DeepSRDCF over EAO, it runs at 50 fps while others runs at 1 fps or below.”
A Twofold Siamese Network for Real-Time Object Tracking (2018, background): “TCNN[28] maintains multiple CNNs in a tree structure to learn ensemble models and estimate target states.”
The Visual Object Tracking VOT2016 Challenge Results (2016, background): “Tracker (TCNN) H. Nam, M. Baek, B. Han { namhs09, mooyeol, bhhan } @postech.ac.kr TCNN [96] maintains multiple target appearance models based on CNNs in a tree structure to preserve model consistency and handle appearance multi-modality eﬀec-tively.”