Deep Extreme Cut: From Extreme Points to Object Segmentation

By Kevis-Kokitsi Maninis, Sergi Caelles, Jordi Pont-Tuset, Luc Van Gool. 2017

462 citations recorded by the source corpus.

This paper explores the use of extreme points in an object (left-most, right-most, top, bottom pixels) as input to obtain precise object segmentation for images and videos. We do so by adding an extra channel to the image in the input of a convolutional neural network (CNN), which contains a Gaussian centered in each of the extreme points. The CNN learns to transform this information into a segmentation of an object that matches those extreme points. We demonstrate the usefulness of this approach for guided segmentation (grabcut-style), interactive segmentation, video object segmentation, and dense segmentation annotation. We show that we obtain the most precise results to date, also with less user input, in an extensive and varied selection of benchmarks and datasets. All our models and code are publicly available on http://www.vision.ee.ethz.ch/~cvlsegmentation/dextr/.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, background): “DEXTR [36] extracts a target object by specifying its four extreme points (left-most, right-most, top, and bottom pixels).”
Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, background): “Binary cross entropy (BCE) loss is a common choice in interactive segmentation [1, 13, 14]; however, it treats almost correctly segmented areas and erroneous regions equally.”
Getting to 99% Accuracy in Interactive Segmentation (2020, methodology): “To integrate the click maps into the network, most papers (iFCN [6], RIS-Net [13], ITIS [7], DEXTR [8]) adopt an early fusion scheme by simply joining the click maps as extra channels to the input image tensor.”
Simitate: A Hybrid Imitation Learning Benchmark (2019, positive): “The images have been labeled with support of a recent guided image segmentation approach [47].”
Bottom-Up Object Detection by Grouping Extreme and Center Points (2019, methodology): “Deep Extreme Cut Deep Extreme Cut (DEXTR) [29] is an extreme point guided image segmentation method.”
Fast Interactive Object Annotation With Curve-GCN (2019, methodology): “Recently, DEXTR [23] incorporated user clicks by stacking them as additional heatmap channels to image features, and exploited the powerful Deeplab architecture to perform user-guided segmentation.”
BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations (2022, background): “Reducing labeling costs can be achieved in a variety of ways, such as interactive humanin-the-loop annotation [2, 50, 53, 64, 73], nearest-neighbor mask transfer [26, 51, 63, 70], or using weak supervision in the form of cheaper labels such as object boxes [7, 43, 45], scribbles [48, 69] or very coa”
Interactive Object Segmentation With Inside-Outside Guidance (2022, methodology): “Compared with existing click-based [1], [37] and box-based [35] interactive segmentation approaches, our proposed IOG has the best of both worlds: (i) ﬂexibility : since the annotated three points are encoded as foreground and background clicks, our IOG naturally supports additional clicks…”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “For the former, the interactive modes are widely explored and mainly based on the bounding box [50], the polygon [1, 6, 32], clicks [2, 29, 36], scribbles [3, 48], and some combinations [34, 52].”
Content-Aware Multi-Level Guidance for Interactive Instance Segmentation (2019, methodology): “Other methods include Graph Cuts [7], Random Walk [18] and GeoS [13] though more recent methods [32, 35, 36, 52, 53] approach the problem with deep learning architectures such as convolutional neural networks (CNNs).”
Deep Interactive Thin Object Selection (2021, methodology): “Similar to [32], our context stream takes four extreme points (top, bottom, leftmost and rightmost pixels) as inputs and converts them to Gaussian heatmaps before concatenating with the image.”
F-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation (2020, background): “Interactive segmentation has been a topic of research for a long time [26, 10, 11, 13, 2, 31, 18, 22, 15].”
Structured Consistency Loss for semi-supervised semantic segmentation (2020, unknown): “Recently, a various techniques such as active learning [15], interactive segmentation [16], weakly-supervised leaning [11] and so on have been developed to solve the labeling cost problem in semantic segmentation.”
Understanding Deep Learning Techniques for Image Segmentation (2019, methodology): “alternative approach follows the footsteps of FCN to fuse features of multiple scales to obtain a sharper resolution image. 4.6.2 Deep Extreme Cut Contrary to the two stream approach, deep extreme cut[138] takes a single pipeline to create segmentation maps from RGB Images. This method expects 4 poi”