Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs

By Ziyu Zhang, Sanja Fidler, Raquel Urtasun. 2015

222 citations recorded by the source corpus.

Our aim is to provide a pixel-wise instance-level labeling of a monocular image in the context of autonomous driving. We build on recent work [32] that trained a convolutional neural net to predict instance labeling in local image patches, extracted exhaustively in a stride from an image. A simple Markov random field model using several heuristics was then proposed in [32] to derive a globally consistent instance labeling of the image. In this paper, we formulate the global labeling problem with a novel densely connected Markov random field and show how to encode various intuitive potentials in a way that is amenable to efficient mean field inference [15]. Our potentials encode the compatibility between the global labeling and the patch-level predictions, contrast-sensitive smoothness as well as the fact that separate regions form different instances. Our experiments on the challenging KITTI benchmark [8] demonstrate that our method achieves a significant performance boost over the baseline [32].

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, background): “The works of [50] and [51] order instances by depth to break this symmetry.”
Hybrid Task Cascade for Instance Segmentation (2019, neutral): “tion predictions. Segmentation-based methods, on the contrary, ﬁrst obtains a pixel-level segmentation map over the image, and then identiﬁes object instances therefrom. Along this line, Zhang et al. [43,42] propose to predict instance labels based on local patches and integrate the local results wi”
Path Aggregation Network for Instance Segmentation (2018, methodology): “A graphical model was used in [66, 65] to infer the order of instances.”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, background): “We note that there are several instance segmentation works [77, 68, 76, 2, 47, 35, 55, 20, 17, 43, 37, 30, 50, 53, 6], which could be potentially extended to bottom-up panoptic segmentation.”
Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning (2019, methodology): “Proposalbased methods [30, 31, 7, 6] predict object masks based on the generated region proposals while image-based methods [47, 48, 44, 2] produce a pixel-level segmentation map over the image to identify object instance.”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, neutral): “[77] propose a novel fully connected CRF [39] (with fast inference by permutohedral lattice [2]) to refine the results.”
Playing for Benchmarks (2017, background): “…ground truth is sparse and is based on ﬁt-ting approximate CAD models to moving objects [42], object detection and segmentation data is available for only a small number of frames and a small set of classes [63], and the dataset as a whole represents the appearance of a single town in fair weather.”
Attention-Guided Unified Network for Panoptic Segmentation (2018, methodology): “FCIS [21], Mask RCNN [12] and PANet [25]. Moreover, segmentation-based methods aggregated pixel-level cues to compose instances based on pre-computed semantic segmentation [2,22,26] or depth ordering [37] results. Semantic Segmentation: With the development of socalled encoding-decoding networks suc”
DeeperLab: Single-Shot Image Parser (2019, methodology): “Another line of work [92, 74, 76, 93, 78] adopts the region-based Covering metric to evaluate the instance segmentation results, which is applicable to methods that do not allow overlapped predictions.”
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++ (2018, background): “Most approaches to object instance segmentation [16, 29, 39, 37, 21, 22, 12, 1, 18] operate on the pixel-level.”
PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (2020, unknown): “[56, 55] utilize MRF for local patch merging.”
PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model (2018, background): “[54,55] predict instance ID by encoding the object depth ordering within a patch and use this depth ordering to cluster instances.”
Weakly Supervised Instance Segmentation Using Class Peak Response (2018, background): “Compared with many fully supervised approaches that typically use complex frameworks including conditional random ﬁelds (CRF) [46, 45], recurrent neural networks (RNN) [30, 32], or template matching [37], to handle instance extraction; our approach is simple yet effective.”
PolyTransform: Deep Polygon Transformer for Instance Segmentation (2019, background): “[66, 65] ﬁrst predicts instance labels within the extracted multi-scale patches and then exploits dense Conditional Random Field [33] to obtain a consistent labeling of the full image.”