Deep Snake for Real-Time Instance Segmentation

By Sida Peng, W. Jiang, Huaijin Pi, Hujun Bao, Xiaowei Zhou. 2020

363 citations recorded by the source corpus.

This paper introduces a novel contour-based approach named deep snake for real-time instance segmentation. Unlike some recent methods that directly regress the coordinates of the object boundary points from an image, deep snake uses a neural network to iteratively deform an initial contour to match the object boundary, which implements the classic idea of snake algorithms with a learning-based approach. For structured feature learning on the contour, we propose to use circular convolution in deep snake, which better exploits the cycle-graph structure of a contour compared against generic graph convolution. Based on deep snake, we develop a two-stage pipeline for instance segmentation: initial contour proposal and contour deformation, which can handle errors in object localization. Experiments show that the proposed approach achieves competitive performances on the Cityscapes, KINS, SBD and COCO datasets while being efficient for real-time applications with a speed of 32.3 fps for 512 x 512 images on a 1080Ti GPU. The code is available at https://github.com/zju3dv/snake/.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

K-Net: Towards Unified Image Segmentation (2021, background): “Some studies explore polar [60] representation, contour [45], and explicit shape representation [62] of instance masks.”
MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction (2022, background): “Deepsnake (Peng et al., 2020) proposes a two-stage contour evolution process and designs circular convolution to exploit the features on the contour.”
End-to-End Instance Edge Detection (2022, background): “Since then, researchers have explored various directions to improve efficiency [4,50] and effectiveness [29,31,13].”
Box-supervised Instance Segmentation with Level Set Evolution (2022, methodology): “DeepSnake [38] is based on the classical snake method [19].”
SOLQ: Segmenting Objects by Learning Queries (2021, background): “Some recent works [8, 9, 10, 11] build instance segmentation frameworks on top of anchor-free object detectors [12, 13] to remove the ROI-Cropping operation, reducing the effect of feature misalignment.”
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation (2020, methodology): “The top-down approaches are divided into two-stage [21,31,8,23,17] and single-stage [13,47,2,42,36] methods.”
Unseen Object Instance Segmentation for Robotic Environments (2020, methodology): “Other methods examine dense sliding-window instance segmentation on 4D tensors [41], combining top-down and bottom-up methods via blending modules [42], and alternative mask representations such as contours [43].”