Content-Aware Multi-Level Guidance for Interactive Instance Segmentation

By Soumajit Majumder, Angela Yao. 2019

90 citations recorded by the source corpus.

In interactive instance segmentation, users give feedback to iteratively refine segmentation masks. The user-provided clicks are transformed into guidance maps which provide the network with necessary cues on the whereabouts of the object of interest. Guidance maps used in current systems are purely distance-based and are either too localized or non-informative. We propose a novel transformation of user clicks to generate content-aware guidance maps that leverage the hierarchical structural information present in an image. Using our guidance maps, even the most basic FCNs are able to outperform existing approaches that require state-of-the-art segmentation networks pre-trained on large scale segmentation datasets. We demonstrate the effectiveness of our proposed transformation strategy through comprehensive experimentation in which we significantly raise state-of-the-art on four standard interactive segmentation benchmarks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “Besides, some state-of-the-art interactive segmentation methods [2, 21, 36, 8, 25] adopt it for training the network.”
Getting to 99% Accuracy in Interactive Segmentation (2020, methodology): “Such click inputs are encoded as images either via a distance transform [6, 10, 12, 13, 18] or by ﬁtting Gaussian masks to each of the clicks [8, 9, 11, 35].”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “[35] improve the transformation of user clicks by generating content-aware guidance maps.”
Deep Interactive Thin Object Selection (2021, methodology): “On this task, deep learning based methods [51, 23, 50, 32, 21, 15, 31, 16, 27, 41] have been very successful.”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, methodology): “Binary cross entropy (BCE) loss is widely used to supervise the interactive segmentation tasks [32,33,34,13,35].”
Interactive Image Segmentation With First Click Attention (2020, methodology): “We also employ the same iterative training strategy [36, 37], which improves the effect of the final model to a certain extent.”
Conditional Diffusion for Interactive Segmentation (2021, methodology): “In a standard pipeline, the first step is encoding clicks into distance maps [32, 27, 11, 31], Gaussian maps [20, 23, 21, 14], or super-pixels [22]; next, encoded maps is concatenated with the original image and fed into a segmentation network to make predictions.”
PolyTransform: Deep Polygon Transformer for Instance Segmentation (2019, background): “[45] uses user clicks to generate content aware guidance maps; all of these help the networks learn stronger cues to generate more accurate segmentations.”
F-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation (2020, background): “Recent works propose different solutions to these problems [19, 18, 21].”
Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections (2019, methodology): “We sample 10 objects per class from the validation set and separately report results for seen (200 objects) and unseen classes (600 objects) as in [56,39].”