Fast Interactive Object Annotation With Curve-GCN

By Huan Ling, Jun Gao, Amlan Kar, Wenzheng Chen, Sanja Fidler. 2019

287 citations recorded by the source corpus.

Manually labeling objects by tracing their boundaries is a laborious process. In Polygon-RNN++, the authors proposed Polygon-RNN that produces polygonal annotations in a recurrent manner using a CNN-RNN architecture, allowing interactive correction via humans-in-the-loop. We propose a new framework that alleviates the sequential nature of Polygon-RNN, by predicting all vertices simultaneously using a Graph Convolutional Network (GCN). Our model is trained end-to-end, and runs in real time. It supports object annotation by either polygons or splines, facilitating labeling efficiency for both line-based and curved objects. We show that Curve-GCN outperforms all existing approaches in automatic mode, including the powerful DeepLab, and is significantly more efficient in interactive mode than Polygon-RNN++. Our model runs at 29.3ms in automatic, and 2.6ms in interactive mode, making it 10x and 100x faster than Polygon-RNN++.

Subfields: Computer Vision and Pattern Recognition, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Graph Learning

Citations in the corpus

EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations (2022, methodology): “To reduce the cost of pixel-level annotation, we use TORonto Annotation Suite (TORAS) [20], which uses interactive tools built from variants of [24, 42].”
Deepstrip: High-Resolution Boundary Refinement (2020, methodology): “Most research in boundary detection focuses on improving the boundary quality in LR through introducing more semantic information [2, 44, 26] or human interaction [22, 25, 42, 24, 7].”
Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis (2021, unknown): “We then predict the position offsets ∆ v i and SDF residual values ∆ s ( v i ) for each vertex i in V surf using a Graph Convolutional Network [32] (GCN): (cid:0) (cid:1) where N surf is the total number of vertices in V surf and f ( v i ) is the updated per-vertex feature.”
Interactive Object Segmentation With Inside-Outside Guidance (2020, methodology): “Following the setting in [46], we compare the performances on two benchmarks, i.e .”
Gated-SCNN: Gated Shape CNNs for Semantic Segmentation (2019, background): “For example, one may need to look at the complete and detailed object boundary to get a discriminative encoding of shape [2, 33], while color and texture contain fairly low-level information.”
KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D (2021, methodology): “the per pixel annotation time of individual images, including classical methods [38], [60] and learning based methods [2], [3], [19], [59].”
PolyTransform: Deep Polygon Transformer for Instance Segmentation (2019, methodology): “Unlike [5, 2, 39], our model has no restriction on the number of polygons utilized to describe each object.”
Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections (2019, methodology): “In interactive segmentation the only existing attempt is [1], which performs polygon annotation [15,1,37].”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, background): “For the former, the interactive modes are widely explored and mainly based on the bounding box [50], the polygon [1, 6, 32], clicks [2, 29, 36], scribbles [3, 48], and some combinations [34, 52].”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, unknown): “Curve-GCN [53]CityScapes N/A ResNet-50 2 60.”
Deep Snake for Real-Time Instance Segmentation (2020, unknown): “Inspired by [21, 25, 38], we deal with this problem in an iterative optimization fashion.”
MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction (2022, unknown): “CurveGCN (Ling et al., 2019)”
SegDiff: Image Segmentation with Diffusion Probabilistic Models (2021, methodology): “We compared our method for the Cityscapes dataset with PSPDeepLab [5], Polygon-RNN++ [1], Curve-GCN [29] Deep active contours [14], Segformer-B5 [50] and Stdc1 [11].”
Deep Interactive Thin Object Selection (2021, methodology): “Right: Segmentation prediction by our proposed TOS-Net trained on our ThinObject-5K dataset. annotation [4, 2, 28].”
Interactive Object Segmentation With Inside-Outside Guidance (2022, methodology): “According to the human inputs, the interactive segmentation approached can be roughly divided into click-based [1], [37], [38], [47], [48], [49], [50], [51] and polygon-based [52], [53], [54].”