Deep Watershed Transform for Instance Segmentation

By Min Bai, Raquel Urtasun. 2016

555 citations recorded by the source corpus.

Most contemporary approaches to instance segmentation use complex pipelines involving conditional random fields, recurrent neural networks, object proposals, or template matching schemes. In this paper, we present a simple yet powerful end-to-end convolutional neural network to tackle this task. Our approach combines intuitions from the classical watershed transform and modern deep learning to produce an energy map of the image where object instances are unambiguously represented as energy basins. We then perform a cut at a single energy level to directly yield connected components corresponding to object instances. Our model achieves more than double the performance over the state-of-the-art on the challenging Cityscapes Instance Level Segmentation task.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Mask R-CNN (2017, neutral): “Finally, we observed a bias between the val and test AP, as is also observed from the results of [23, 4].”
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “BBFNet [8] obtains instance segmentation results by Watershed transform [84,4] and Hough-voting [5,51].”
Panoptic Segmentation (2018, methodology): “Methods that generate non-overlapping instance segmentations [18, 3, 28, 2] use the same format as PS, but these methods typically only address thing classes.”
Recurrent Neural Networks for Semantic Instance Segmentation (2017, methodology): “In this paper, we argue that both ranking thousands of object-prominent regions [17, 18, 24, 23, 11, 19] or decomposing the problem into two independent modules [6, 12, 34] are unnatural approaches that are far away from the sequential exploration that we as humans follow to solve related tasks such”
K-Net: Towards Unified Image Segmentation (2021, methodology): “‘Bottom-up’ methods [1, 30, 43, 44] first perform semantic segmentation then group pixels into different instances.”
Nucleus segmentation across imaging experiments: The 2018 Data Science Bowl (2019, neutral): “Apart from the top three methods, the fourth place solution 34,35 by the team ‘Nuclear Vision’ fell 0.04 points behind third place and combined classical watershed transform with modern deep learning 36 .”
Hybrid Task Cascade for Instance Segmentation (2019, background): “Bai and Urtasun [2] propose an alternative way, which combines watershed transform and deep learning to produce an energy map, and then derive the instances by dividing the output of the watershed transform.”
Path Aggregation Network for Instance Segmentation (2018, background): “They learned specially designed transformation [3, 33, 38, 59] or instance boundaries [30].”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Then, instance segments are obtained by grouping ‘thing’ pixels with various methods, such as instance center regression [44, 86, 70, 100, 20], Watershed transform [88, 3, 8], Hough-voting [4, 53, 8], or pixel affinity [45, 66, 81, 30, 8].”
Proposal-Free Network for Instance-Level Object Segmentation (2015, neutral): “Recently, more research interests have been attracted to develop kinds of “proposal-free”-like pipelines [35], [1], [16], [20] that aim to generate instance-level object segmentation without relying on any proposal generation.”
On the Binding Problem in Artificial Neural Networks (2020, methodology): “Other approaches output an energy function from which the segmentation is easily derived, e.g. based on the Watershed transformation (Bai and Urtasun, 2017).”
Incremental Few-Shot Instance Segmentation (2021, neutral): “Approaches generally fall into two categories: grouping-based [4, 11, 17, 24] and proposal-based [2, 6, 15, 20] detection methods.”
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics (2017, methodology): “Methods which rely on watershed approaches [4], or instance edge identification approaches fail in these scenarios.”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, methodology): “We note that there are several instance segmentation works [77, 68, 76, 2, 48, 35, 56, 20, 18, 44, 37, 30, 51, 54, 6], which could be potentially extended to bottom-up panoptic segmentation.”
Mask Scoring R-CNN (2019, methodology): “[1] predicted pixel-level energy values and used watershed algorithms for grouping.”
YOLACT: Real-Time Instance Segmentation (2019, methodology): “Finally, some methods first perform semantic segmentation followed by boundary detection [22], pixel clustering [3, 25], or learn an embedding to form instance masks [32, 17, 9, 13].”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, methodology): “[72] exploit a localization network for grouping, while Bai and Urtasun [6] adopt a Watershed Transform Net.”
Attention-Guided Unified Network for Panoptic Segmentation (2018, neutral): “DWT [3] VGG16 - - - 21.”