Fully-Convolutional Siamese Networks for Object Tracking
By Luca Bertinetto, Jack Valmadre, João F. Henriques, Andrea Vedaldi, Philip H. S. Torr. 2016
4,373 citations recorded by the source corpus.
The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object’s appearance exclusively online, using as sole training data the video itself. Despite the success of these methods, their online-only approach inherently limits the richness of the model they can learn. Recently, several attempts have been made to exploit the expressive power of deep convolutional networks. However, when the object to track is not known beforehand, it is necessary to perform Stochastic Gradient Descent online to adapt the weights of the network, severely compromising the speed of the system. In this paper we equip a basic tracking algorithm with a novel fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset for object detection in video. Our tracker operates at frame-rates beyond real-time and, despite its extreme simplicity, achieves state-of-the-art performance in multiple benchmarks.
Subfields: Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Yi Yang
- Andrew Zisserman
- Jitendra Malik
In Safety, Risk, Reliability and Quality
Citations in the corpus
- Exploring Simple Siamese Representation Learning (2020, background): “Their applications include signature [4] and face [34] verification, tracking [3], one-shot learning [23], and others.”
- vCLIMB: A Novel Video Class Incremental Learning Benchmark (2022, background): “A significant result of such pretraining is enabling feature reuse [12,41], by means of finetuning the learned weights for smaller downstream tasks [3,22,28].”
- Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, methodology): “This operation is analogous to the search function in visual tracking literature [35, 36, 37] where ot[Tpick] is the exemplar and ot is the search area, or in retrieval systems literature where ot[Tpick] is the query and ot is the key.”
- Practical Network Blocks Design with Q-Learning (2017, methodology): “…and Hinton 2012; Simonyan and Zisserman 2014; Szegedy et al. 2015a; He et al. 2016a), and then be applied to object detection (Gir-shick 2015; Ren et al. 2015), semantic segmentation (Long, Shelhamer, and Darrell 2015; Chen et al. 2016) and tracking (Nam and Han 2016; Bertinetto et al. 2016).”
- Neural Snowball for Few-Shot Relation Learning (2019, background): “, 2018), object tracking (Bertinetto et al., 2016) and measuring text similarities (Neculoiu et al.”
- Video Instance Segmentation (2019, background): “The other setting is the detectionfree tracking [1, 19, 9], which targets at tracking objects given their initial bounding boxes in the first frame.”
- Learning Video Object Segmentation From Unlabeled Videos (2020, neutral): “With a design similar to the classic Siamese tracker [2], we forward track the patch p on the next frame Xt+1 by conducting a crosscorrelation operation ‘⋆’ on φ(p) and φ(Xt+1): S⇒ = φ(p) ⋆ φ(Xt+1) ∈ [0, 1] W×H , (3)”
- Deeper and Wider Siamese Networks for Real-Time Visual Tracking (2019, methodology): “0%(EAO) relative improvements over the original versions [2, 20] on the OTB-15, VOT-16 and VOT-17 datasets, respectively.”
- Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking (2022, methodology): “Concretely, SiameseFC [1] utilize the response map from cross-correlation between the exemplar and search features for target localization.”
- Siam R-CNN: Visual Tracking by Re-Detection (2019, background): “In the past, the problem of distractors has mainly been approached by strong spatial priors from previous predictions [4, 38, 37], or by online adaptation [1, 21, 2, 56, 23, 57, 32].”
- CREST: Convolutional Residual Learning for Visual Tracking (2017, methodology): “We compare our CREST tracker with 29 trackers from the OTB-2013 benchmark [45] and other 21 state-of-the-art trackers including KCF [17], MEEM [48], TGPR [13], DSST [7], RPT [27], MUSTer [20], LCT [31], HCFT [30], FCNT [42], SRDCF [9], CNN-SVM [19], DeepSRDCF [8], DAT [35], Staple [2], SRDCFdecon [1”
- Deformable Siamese Attention Networks for Visual Object Tracking (2020, methodology): “[1] first introduced SiamFC for visual tracking, by using Siamese networks to measure the similarity between target and search image.”
- Distractor-aware Siamese Networks for Visual Object Tracking (2018, methodology): “As a follow-up work, Bertinetto et.al [2] propose a fully convolutional Siamese network (SiamFC) to estimate the feature similarity region-wise between two frames.”
- Discriminative Correlation Filter Tracker with Channel and Spatial Reliability (2016, methodology): “The set is indeed diverse, the top-performing trackers come from various classes e.g., correlation filter methods (CCOT [13], Staple [2], DDC [25]), deep ConvNets (TCNN [25], SSAT[25, 37], MLDF [25, 40], FastSiamnet [3]) and different detection-Overall based approaches (EBT [47], SRBT [25]).”