SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation
By Dongfang Liu, Yiming Cui, Wenbo Tan, Yingjie Chen. 2021
220 citations recorded by the source corpus.
Video instance segmentation (VIS) is a new and critical task in computer vision. To date, top-performing VIS methods extend the two-stage Mask R-CNN by adding a tracking branch, leaving plenty of room for improvement. In contrast, we approach the VIS task from a new perspective and propose a one-stage spatial granularity network (SG-Net). Compared to the conventional two-stage methods, SG-Net demonstrates four advantages: 1) Our method has a one-stage compact architecture and each task head (detection, segmentation, and tracking) is crafted interdependently so they can effectively share features and enjoy the joint optimization; 2) Our mask prediction is dynamically performed on the sub-regions of each detected instance, leading to high-quality masks of fine granularity; 3) Each of our task predictions avoids using expensive proposal-based RoI features, resulting in much reduced runtime complexity per instance; 4) Our tracking head models objects’ centerness movements for tracking, which effectively enhances the tracking robustness to different object appearances. In evaluation, we present state-of-the-art comparisons on the YouTube-VIS dataset. Extensive experiments demonstrate that our compact one-stage method can achieve improved performance in both accuracy and inference speed. We hope our SG-Net could serve as a strong and flexible base-line for the VIS task. Our code will be available here 1.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Alan L. Yuille
- Philip H. S. Torr
- Xiaogang Wang
- Yi Yang
- Dacheng Tao
- Chunhua Shen
Citations in the corpus
- Occluded Video Instance Segmentation: A Benchmark (2021, background): “, 2019), SGNet (Liu et al., 2021) dynamically divides instances into sub-regions and performs segmentation on each region.”
- Video Instance Segmentation using Inter-Frame Communication Transformers (2021, background): “Most of the current approaches [1, 2, 3, 4] extend image instance segmentation models [5, 6, 7, 8] and take frame-wise inputs.”
- A Survey on Deep Learning Technique for Video Segmentation (2021, background): “Alternatively, [237] models spatial attention to describe instances, tackling the task from a novel singlestage yet elegant perspective.”