Deep GrabCut for Object Selection

By Ning Xu, Brian L. Price, Scott D. Cohen, Jimei Yang, Thomas S. Huang. 2017

136 citations recorded by the source corpus.

Most previous bounding-box-based segmentation methods assume the bounding box tightly covers the object of interest. However it is common that a rectangle input could be too large or too small. In this paper, we propose a novel segmentation approach that uses a rectangle as a soft constraint by transforming it into an Euclidean distance map. A convolutional encoder-decoder network is trained end-to-end by concatenating images with these distance maps as inputs and predicting the object masks as outputs. Our approach gets accurate segmentation results given sloppy rectangles while being general for both interactive segmentation and instance segmentation. We show our network extends to curve-based input without retraining. We further apply our network to instance-level semantic segmentation and resolve any overlap using a conditional random field. Experiments on benchmark datasets demonstrate the effectiveness of the proposed approaches.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, background): “ConvNet-based methods have explored various interaction types, such as bounding boxes [47], polygons [1], clicks [43], and scribbles [45].”
Deep Extreme Cut: From Extreme Points to Object Segmentation (2017, methodology): “Interactive User Input: The pipeline of DEXTR can also be used in the frame of interactive segmentation from points [41, 40].”
Interactive Object Segmentation With Inside-Outside Guidance (2020, methodology): “Interactive Segmentation: Most interactive segmentation methods target at relieving human effort when labeling pixel-level annotations with the help of bounding box [66, 64, 52], clicks [67, 38, 37, 45] or contours [7, 1, 42] as guidance.”
Content-Aware Multi-Level Guidance for Interactive Instance Segmentation (2019, methodology): “Other methods include Graph Cuts [7], Random Walk [18] and GeoS [13] though more recent methods [32, 35, 36, 52, 53] approach the problem with deep learning architectures such as convolutional neural networks (CNNs).”
Interactive Object Segmentation With Inside-Outside Guidance (2022, methodology): “Compared with existing click-based [1], [37] and boxbased [35] interactive segmentation approaches, our proposed IOG has the best of both worlds: (i) flexibility: since the annotated three points are encoded as foreground and background clicks, our IOG naturally supports additional clicks annotation”
Large-Scale Interactive Object Segmentation With Human Annotators (2019, background): “Despite the theme of human-machine interaction, most previous work in this area report purely simulation experiments [35, 17, 20, 22] or only small scale human experiments (≤ 50 instances [15, 16, 1]).”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, background): “For the former, the interactive modes are widely explored and mainly based on the bounding box [50], the polygon [1, 6, 32], clicks [2, 29, 36], scribbles [3, 48], and some combinations [34, 52].”
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click (2020, methodology): “These inputs usually come in the form of user clicks/strokes [5,28,34,41,51,58] or bounding boxes [16, 29, 49, 57].”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, methodology): “While different interaction types have been investigated, including clicks [10,11], bounding boxes [12,13,14], and scribbles [15,16], we only focus on click-based interactive segmentation because it has the simplest interaction and well-established ar X iv :2 20 7.”
Deep Interactive Thin Object Selection (2021, methodology): “On this task, deep learning based methods [51, 23, 50, 32, 21, 15, 31, 16, 27, 41] have been very successful.”
A Fully Convolutional Two-Stream Fusion Network for Interactive Image Segmentation (2018, background): “Xu et al. [18] propose Deep GrabCut.”
YouTube-VOS: Sequence-to-Sequence Video Object Segmentation (2018, unknown): “great success in image segmentation tasks [36,7,49,48].”
Conditional Diffusion for Interactive Segmentation (2021, unknown): “In a standard pipeline, the first step is encoding clicks into distance maps [32, 27, 11, 31], Gaussian maps [20, 23, 21, 14], or super-pixels [22]; next, encoded maps is concatenated with the original image and fed into a segmentation network to make predictions.”