InstanceCut: From Edges to Instances with MultiCut

By Alexander Kirillov, Evgeny Levinkov, Bjoern Andres, Bogdan Savchynskyy, Carsten Rother. 2016

261 citations recorded by the source corpus.

This work addresses the task of instance-aware semantic segmentation. Our key motivation is to design a simple method with a new modelling-paradigm, which therefore has a different trade-off between advantages and disadvantages compared to known approaches. Our approach, we term InstanceCut, represents the problem by two output modalities: (i) an instance-agnostic semantic segmentation and (ii) all instance-boundaries. The former is computed from a standard convolutional neural network for semantic segmentation, and the latter is derived from a new instance-aware edge detection model. To reason globally about the optimal partitioning of an image into instances, we combine these two modalities into a novel MultiCut formulation. We evaluate our approach on the challenging CityScapes dataset. Despite the conceptual simplicity of our approach, we achieve the best result among all published methods, and perform particularly well for rare object classes.

Subfields: Computer Vision and Pattern Recognition, Ocean Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Ocean Engineering

Citations in the corpus

Panoptic Feature Pyramid Networks (2019, methodology): “, [31, 1, 38] use a separate network to predict instance edges, bounding boxes, and object breakpoints, respectively).”
Panoptic Segmentation (2018, methodology): “Methods that generate non-overlapping instance segmentations [18, 3, 28, 2] use the same format as PS, but these methods typically only address thing classes.”
K-Net: Towards Unified Image Segmentation (2021, methodology): “‘Bottom-up’ methods [1, 30, 43, 44] first perform semantic segmentation then group pixels into different instances.”
Path Aggregation Network for Instance Segmentation (2018, background): “They learned specially designed transformation [3, 33, 38, 59] or instance boundaries [30].”
Proposal-Free Network for Instance-Level Object Segmentation (2015, neutral): “Recently, more research interests have been attracted to develop kinds of “proposal-free”-like pipelines [35], [1], [16], [20] that aim to generate instance-level object segmentation without relying on any proposal generation.”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, background): “We note that there are several instance segmentation works [77, 68, 76, 2, 48, 35, 56, 20, 18, 44, 37, 30, 51, 54, 6], which could be potentially extended to bottom-up panoptic segmentation.”
Mask Scoring R-CNN (2019, background): “Other works, such as [20, 21], add boundary detection information during the clustering procedure.”
YOLACT: Real-Time Instance Segmentation (2019, methodology): “Finally, some methods first perform semantic segmentation followed by boundary detection [22], pixel clustering [3, 25], or learn an embedding to form instance masks [32, 17, 9, 13].”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, background): “[38, 36] exploit boundary detection information, while [55, 23, 8] propose to cluster instances w.”
Semantic Instance Segmentation with a Discriminative Loss Function (2017, methodology): “[20] use a CRF, but with a novel MultiCut formulation to combine semantic segmentations with edge maps to extract instances as connected regions.”
Recurrent Pixel Embedding for Instance Grouping (2017, neutral): “, [40]) but typically don’t address end-to-end learning.”
DeeperLab: Single-Shot Image Parser (2019, methodology): “On the other hand, the bottom-up approaches [73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84] generally adopt a two-stage processing: pixel-level predictions produced by the segmentation module are clustered together to form the instancelevel predictions.”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, neutral): “5 InstanceCut [30] – – – – – – – – – 13.”
MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation (2020, background): “It segments an image into semantically meaningful instances [44], [45], [46], [47], [48].”