Learning Multi-domain Convolutional Neural Networks for Visual Tracking

By Hyeonseob Nam, Bohyung Han. 2015

2,617 citations recorded by the source corpus.

We propose a novel visual tracking algorithm based on the representations from a discriminatively trained Convolutional Neural Network (CNN). Our algorithm pretrains a CNN using a large set of videos with tracking ground-truths to obtain a generic target representation. Our network is composed of shared layers and multiple branches of domain-specific layers, where domains correspond to individual training sequences and each branch is responsible for binary classification to identify target in each domain. We train each domain in the network iteratively to obtain generic target representations in the shared layers. When tracking a target in a new sequence, we construct a new network by combining the shared layers in the pretrained CNN with a new binary classification layer, which is updated online. Online tracking is performed by evaluating the candidate windows randomly sampled around the previous target state. The proposed algorithm illustrates outstanding performance in existing tracking benchmarks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “Many-to-one multilingual NMT can be categorized as a multi-domain 3 learning problem (Dredze et al., 2010; Joshi et al., 2012; Nam and Han, 2015), where the task of translating into a selected language remains the same, but the input distribution is different across source languages (Arivazhagan et…”
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, background): “tection [29, 23], semantic segmentation [1, 24], and video analysis [27, 31].”
Learning feed-forward one-shot learners (2016, background): “We remark that the top-ranking tracker, MDNet [15], ﬁne-tunes a network with SGD during tracking, which is computationally expensive (1 frame per second on a GPU), while all our networks operate in feed-forward mode during tracking and run at at least 60 FPS.”
Learning without Forgetting (2016, methodology): “We build on MD-Net [35], which views tracking as a template classification task.”
Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation (2022, background): “Deep neural networks have achieved outstanding results in various applications including computer vision [13, 29, 31, 32, 38], natural language processing [45, 47], speech recognition [5, 10], robotics [25], bioinformatics [6], and many others.”
Practical Network Blocks Design with Q-Learning (2017, methodology): “…and Hinton 2012; Simonyan and Zisserman 2014; Szegedy et al. 2015a; He et al. 2016a), and then be applied to object detection (Gir-shick 2015; Ren et al. 2015), semantic segmentation (Long, Shelhamer, and Darrell 2015; Chen et al. 2016) and tracking (Nam and Han 2016; Bertinetto et al. 2016).”
Fully-Convolutional Siamese Networks for Object Tracking (2016, methodology): “All of the competitive methods above that train on video sequences (MDNet [9], SINT [30], GOTURN [28]), use training data belonging to the same ALOV/OTB/VOT domain used by the benchmarks.”
One-Shot Video Object Segmentation (2016, background): “We can see that OSVOS has significant better results as tracker than MDNET at all regimes, with more margin at higher thresholds.”
Learning Video Object Segmentation from Static Images (2016, methodology): “MDNet [32] proposes to use online fine-tuning of a convnet to model the object appearance.”
Video Instance Segmentation (2019, background): “The other setting is the detectionfree tracking [1, 19, 9], which targets at tracking objects given their initial bounding boxes in the first frame.”
End-to-end Recurrent Multi-Object Tracking and Trajectory Prediction with Relational Reasoning (2019, background): “Recently, elements of this pipeline have been replaced with learning-based approaches such as deep learning [15, 16, 4, 3] or reinforcement learning [17].”
BlockQNN: Efficient Block-Wise Neural Network Architecture Generation (2018, background): “tion [3], [4], semantic segmentation [5], [6] and tracking [7], [8].”
Transformer Tracking (2021, methodology): “In this subsection, we compare our TransT method with twelve state-of-the-art trackers published in 2020 (SiamR-CNN [39], Ocean [48], KYS [3], DCFST [49], SiamFC++ [44], PrDiMP [10], CGACD [13], Sia-mAttn [46], MAML [40], D3S [26], SiamCAR [16], and SiamBAN [5]) and six representative trackers prese”
Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning (2017, methodology): “Recently, tracking methods [35, 14, 24] using convolutional neural networks (CNNs) have been proposed for robust tracking and vastly improved tracking performance with the help of rich feature representation.”
SCALOR: Generative World Models with Scalable Object Representations (2019, methodology): “…computer vision community using the same “bounding box”-representation approach proposed in SCALOR (Kosiorek et al., 2017; Ning et al., 2017; Nam & Han, 2016; Tao et al., 2016), but such methods use provided labels to tackle the problem of object tracking and thus are usually fully or…”
Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking (2018, methodology): “We compare STRCF and DeepSTRCF with state-of-theart trackers, including MDNet [32] (VOT-2015 winner) and TCNN [31] (VOT-2016 winner).”
Automatic Adaptation of Object Detectors to New Domains Using Self-Training (2019, background): “3 of [26]) with the MD-Net tracker [38].”