MultiSeg: Semantically Meaningful, Scale-Diverse Segmentations From Minimal User Input

By J. Liew, Scott D. Cohen, Brian L. Price, Long Mai, Sim Heng Ong, Jiashi Feng. 2019

40 citations recorded by the source corpus.

Existing deep learning-based interactive image segmentation approaches typically assume the target-of-interest is always a single object and fail to account for the potential diversity in user expectations, thus requiring excessive user input when it comes to segmenting an object part or a group of objects instead. Motivated by the observation that the object part, full object, and a collection of objects essentially differ in size, we propose a new concept called scale-diversity, which characterizes the spectrum of segmentations w.r.t. different scales. To address this, we present MultiSeg, a scale-diverse interactive image segmentation network that incorporates a set of two-dimensional scale priors into the model to generate a set of scale-varying proposals that conform to the user input. We explicitly encourage segmentation diversity during training by synthesizing diverse training samples for a given image. As a result, our method allows the user to quickly locate the closest segmentation target for further refinement if necessary. Despite its simplicity, experimental results demonstrate that our proposed model is capable of quickly producing diverse yet plausible segmentation outputs, reducing the user interaction required, especially in cases where many types of segmentations (object parts or groups) are expected.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

FocalClick: Towards Practical Interactive Image Segmentation (2022, background): “After this, [22,25] focus on the issue of ambiguity and predict multiple potential results and let a selection network or the user choose from them.”
Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “The vast majority of recent CNN-based interactive segmentation methods [2, 19, 20, 21, 36, 4, 1] are trained either using Semantic Boundaries Dataset [39], or the Pascal VOC dataset [40], or the combination of these two datasets, as they share the same set of images.”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, methodology): “Binary cross entropy (BCE) loss is widely used to supervise the interactive segmentation tasks [32,33,34,13,35].”
Deep Interactive Thin Object Selection (2021, background): “[22] addressed the ambiguity in interactive segmentation by enabling multiple hypothesis segmentation.”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “[30] introduce scale diversity into the model to help users quickly locate their desired target.”
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click (2020, background): “Furthermore, to encourage the model to learn to generate correct prediction from ambiguous clicks, we follow [33] and introduce another sampling strategy that samples clicks on the overlapped foreground and overlapped background of diﬀerent object masks, for example, the positive and negative…”