Object Instance Annotation With Deep Extreme Level Set Evolution

By Zian Wang, David Acuna, Huan Ling, Amlan Kar, Sanja Fidler. 2019

87 citations recorded by the source corpus.

In this paper, we tackle the task of interactive object segmentation. We revive the old ideas on level set segmentation which framed object annotation as curve evolution. Carefully designed energy functions ensured that the curve was well aligned with image boundaries, and generally "well behaved". The Level Set Method can handle objects with complex shapes and topological changes such as merging and splitting, thus able to deal with occluded objects and objects with holes. We propose Deep Extreme Level Set Evolution that combines powerful CNN models with level set optimization in an end-to-end fashion. Our method learns to predict evolution parameters conditioned on the image and evolves the predicted initial contour to produce the final result. We make our model interactive by incorporating user clicks on the extreme boundary points, following DEXTR. We show that our approach significantly outperforms DEXTR on the static Cityscapes dataset and the video segmentation benchmark DAVIS, and performs on par on PASCAL and SBD.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations (2022, methodology): “To reduce the cost of pixel-level annotation, we use TORonto Annotation Suite (TORAS) [20], which uses interactive tools built from variants of [24, 42].”
Devil Is in the Edges: Learning Semantic Boundaries From Noisy Annotations (2019, background): “In parallel work, [33] learns CNN feature extraction and levelset evolution in an end-to-end fashion for object instance annotation.”
PolyTransform: Deep Polygon Transformer for Instance Segmentation (2019, methodology): “In the context of interactive annotation, we out-perform the previous state of the art [61] by 2.0 % in the boundary metric.”
Boundary IoU: Improving Object-Centric Image Segmentation Evaluation (2021, background): “Both metrics use Mask IoU and inherit its bias toward large objects and, subsequently, the insensitivity to the boundary quality observed before in [22, 35].”
SegFix: Model-Agnostic Boundary Refinement for Segmentation (2020, methodology): “Recent work [1,14,53,31] extended the conventional level-set scheme to deep network for regularizing the boundaries of predicted segmentation map.”
DARNet: Deep Active Ray Network for Building Segmentation (2019, unknown): “[9, 23] propose to combine a CNN with a level set method [16] in an end-to-end differentiable manner.”
Deepstrip: High-Resolution Boundary Refinement (2020, methodology): “For active contour methods, the baselines are Curve-GCN [25] and DELSE [38] , and predictions on Pix-aHR are made in LR and upsampled to original resolution since the whole boundary region is required at inference.”
BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations (2022, background): “Reducing labeling costs can be achieved in a variety of ways, such as interactive humanin-the-loop annotation [2, 50, 53, 64, 73], nearest-neighbor mask transfer [26, 51, 63, 70], or using weak supervision in the form of cheaper labels such as object boxes [7, 43, 45], scribbles [48, 69] or very coa”
Box-supervised Instance Segmentation with Level Set Evolution (2022, methodology): “Some works [15, 17, 21, 49, 54] have been proposed to embed the level set into the deep network in an end-to-end manner and achieve promising segmentation results.”