Bottom-up Instance Segmentation using Deep Higher-Order CRFs

By Anurag Arnab, Philip H. S. Torr. 2016

54 citations recorded by the source corpus.

Traditional Scene Understanding problems such as Object Detection and Semantic Segmentation have made breakthroughs in recent years due to the adoption of deep learning. However, the former task is not able to localise objects at a pixel level, and the latter task has no notion of different instances of objects of the same class. We focus on the task of Instance Segmentation which recognises and localises objects down to a pixel level. Our model is based on a deep neural network trained for semantic segmentation. This network incorporates a Conditional Random Field with end-to-end trainable higher order potentials based on object detector outputs. This allows us to reason about instances from an initial, category-level semantic segmentation. Our simple method effectively leverages the great progress recently made in semantic segmentation and object detection. The accurate instance-level segmentations that our network produces is reflected by the considerable improvements obtained over previous work.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recurrent Neural Networks for Semantic Instance Segmentation (2017, critical): “Compared to more recent works [24, 4, 25, 5], our method falls behind”
Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, methodology): “Other segmentation-based approaches, [3, 28] are not fully endto-end trained.”
Deep Watershed Transform for Instance Segmentation (2016, methodology): “Recent work by [2] uses object detection proposals in conjunction with a deep high order conditional random ﬁeld (CRF) to reason about pixel assignment in overlapping object proposal boxes.”
Hybrid Task Cascade for Instance Segmentation (2019, methodology): “Arnab and Torr [1] also use CRF to identify instances.”
Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning (2019, methodology): “Proposalbased methods [30, 31, 7, 6] predict object masks based on the generated region proposals while image-based methods [47, 48, 44, 2] produce a pixel-level segmentation map over the image to identify object instance.”
Semantic Instance Segmentation with a Discriminative Loss Function (2017, methodology): “Two popular approaches introduce a multi-stage pipeline with object proposals [17, 9, 28, 29, 12, 13, 42, 4], or train a recurrent network end-to-end with a custom loss function that outputs instances sequentially [26, 31, 30].”
Gated-SCNN: Gated Shape CNNs for Semantic Segmentation (2019, background): “Several works [9, 32, 59, 43, 20, 3, 36, 23, 5] propose to use structured prediction modules such as conditional random ﬁelds (CRFs) on network output for improving the segmentation performance, especially around object boundaries.”
BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation (2020, background): “Recognizing these difﬁculties, a line of research takes a bottom-up strategy [1, 19, 20].”
SGN: Sequential Grouping Networks for Instance Segmentation (2017, methodology): “In [1, 2], a CRF is used to assign each pixel to an object detection box by exploiting semantic segmentation maps.”