Fast Online Object Tracking and Segmentation: A Unifying Approach

By Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, Philip H. S. Torr. 2018

1,325 citations recorded by the source corpus.

In this paper we illustrate how to perform both visual object tracking and semi-supervised video object segmentation, in real-time, with a single simple approach. Our method, dubbed SiamMask, improves the offline training procedure of popular fully-convolutional Siamese approaches for object tracking by augmenting their loss with a binary segmentation task. Once trained, SiamMask solely relies on a single bounding box initialisation and operates online, producing class-agnostic object segmentation masks and rotated bounding boxes at 55 frames per second. Despite its simplicity, versatility and fast speed, our strategy allows us to establish a new state-of-the-art among real-time trackers on VOT-2018, while at the same time demonstrating competitive performance and the best speed for the semi-supervised video object segmentation task on DAVIS-2016 and DAVIS-2017.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

EDTER: Edge Detection with Transformer (2022, background): “Edge detection is one of the most fundamental problems in computer vision and has a wide variety of applications, such as image segmentation [8, 23, 39, 44, 45, 47], object detection [23], and video object segmentation [5, 57, 59].”
Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[222] Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, and Philip H.”
Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, background): “Image segmentation is an essential computer vision ability for delivering technologies such as autonomous driving (Zhang, Fidler, and Urtasun 2016) and automatic object tracking (Wang et al. 2019).”
An Empirical Study of Detection-Based Video Instance Segmentation (2019, methodology): “In this work, we exploit the efficiency of the light and robust Siamese tracker, SiamMask [23], for fast tracking and accurate segmentation of objects in videos.”
Artificial intelligence in the creative industries: a review (2020, methodology): “bject in one frame and, by tracking the region, these adjusted parameters can be applied to the rest of the sequence regardless of object motion. Semi-supervised learning is also employed in SiamMask [182] oering the user an interface to dene the object of interest and to track it over over time. Si”
A Survey on Deep Learning Technique for Video Segmentation (2021, methodology): “Core Architecture Flow Technical Feature Training Dataset 2017 OSVOS [18] CVPR FCN Online Fine-tuning DAVIS16 [17] MaskTrack [117] CVPR FCN ✓ Propagation-based ECSSD [118] + MSRA10K [84] + PASCAL- S [88] + DAVIS16 [17] CTN [119] CVPR FCN ✓ Propagation-based PASCAL VOC 2012 [103] VPN [120] CVPR Bilat”
Transformer Tracking (2021, methodology): “Some further improvements have been made, such as adding additional branches [42, 45], using deeper architectures [21], exploiting anchor-free architectures [44, 48], and so on.”
MAST: A Memory-Augmented Self-Supervised Tracker (2020, neutral): “OSVOS-S [42], RGMP [44], RVOS [54], FEELVOS [56], OnAVOS [57], Video Colorization [59], SiamMask [61], CycleTime [64], RANet [65], OSMN [73], with bounding boxes throughout the video; the other aims for more ﬁne-grained tracking, i.e . relocalize the objects with pixel-level segmentation masks, also”
Siamese Box Adaptive Network for Visual Tracking (2020, methodology): “At the same time, in order to improve the receptive field, we use atrous convolution [4], which is proven to be effective for visual tracking [21, 42].”
Is First Person Vision Challenging for Object Tracking? (2020, neutral): “Trackers based on siamese networks [8, 91, 36, 15, 48, 80, 90, 92] emerge as the fastest trackers, thanks to their non-reliance of online adaptation mechanisms.”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “The advent of deep convolutional neural networks [17, 20, 64] has promoted dramatic advances in many computer vision tasks including object tracking [28, 29, 45], visual correspondence [22, 44, 48], and semantic segmentation [7, 47, 62] to name a few.”
Deep Learning for Visual Tracking: A Comprehensive Survey (2019, methodology): “To enhance efficiency and accuracy, the target regions are estimated by classification scores and optimized regression/matching functions [56], [57], [134], [145], [146], [156], [157], [160], [163], [164], [167], [179].”