Interactive Full Image Segmentation by Considering All Regions Jointly

By Eirikur Agustsson, Jasper R. R. Uijlings, Vittorio Ferrari. 2018

79 citations recorded by the source corpus.

We address interactive full image annotation, where the goal is to accurately segment all object and stuff regions in an image. We propose an interactive, scribble-based annotation framework which operates on the whole image to produce segmentations for all regions. This enables sharing scribble corrections across regions, and allows the annotator to focus on the largest errors made by the machine across the whole image. To realize this, we adapt Mask-RCNN [22] into a fast interactive segmentation framework and introduce an instance-aware loss measured at the pixel-level in the full image canvas, which lets predictions for nearby regions properly compete for space. Finally, we compare to interactive single object segmentation on the COCO panoptic dataset [11, 27, 34]. We demonstrate that our interactive full image segmentation approach leads to a 5% IoU gain, reaching 90% IoU at a budget of four extreme clicks and four corrective scribbles per region

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, background): “Interactive segmentation can significantly simplify and speed up this process [9, 10, 11], which is one of the most important applications of interactive segmentation.”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “For the former, the interactive modes are widely explored and mainly based on the bounding box [50], the polygon [1, 6, 32], clicks [2, 29, 36], scribbles [3, 48], and some combinations [34, 52].”
Interactive Object Segmentation With Inside-Outside Guidance (2022, background): “Interactive full image segmentation [80] aims at segmenting all object and stuff regions simultaneously in one image.”
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click (2020, background): “Agustsson et al. [2] further segment all regions jointly with extreme points and scribbles.”
F-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation (2020, methodology): “Annotation of these datasets became feasible with the use of automated interactive segmentation methods [1, 3].”
Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections (2019, methodology): “ssical methods are based on a weak appearance model which is specialized to one speciﬁc image. Recent methods rely on Convolutional Neural Networks (CNNs) to interactively produce a segmentation mask [52, 33,10,35,31,15,27,28,3]. These methods take the image and user corrections (transformed into a ”