Devil Is in the Edges: Learning Semantic Boundaries From Noisy Annotations

By David Acuna, Amlan Kar, Sanja Fidler. 2019

154 citations recorded by the source corpus.

We tackle the problem of semantic boundary prediction, which aims to identify pixels that belong to object(class) boundaries. We notice that relevant datasets consist of a significant level of label noise, reflecting the fact that precise annotations are laborious to get and thus annotators trade-off quality with efficiency. We aim to learn sharp and precise semantic boundaries by explicitly reasoning about annotation noise during training. We propose a simple new layer and loss that can be used with existing learning-based boundary detectors. Our layer/loss enforces the detector to predict a maximum response along the normal direction at an edge, while also regularizing its direction. We further reason about true object boundaries during training using a level set formulation, which allows the network to learn from misaligned labels in an end-to-end fashion. Experiments show that we improve over the CASENet backbone network by more than 4% in terms of MF(ODS) and 18.61% in terms of AP, outperforming all current state-of-the-art methods including those that deal with alignment. Furthermore, we show that our learned network can be used to significantly improve coarse segmentation labels, lending itself as an efficient way to label new data.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Bounding-Box Free Panoptic Segmentation (2020, background): “d exploit it towards our goal of an accurate, bounding-box free panoptic segmentation. Semantic Boundaries: In addition to above, a parallel body of work deals with detection of object boundaries. In [1] the authors focus on detecting accurate object boundaries by explicitly reasoning about annotati”
Fast Interactive Object Annotation With Curve-GCN (2019, methodology): “In [1], levelset evolution with carefully designed boundary prediction was used to find accurate object boundaries from coarse annotations.”
Gated-SCNN: Gated Shape CNNs for Semantic Segmentation (2019, methodology): “We follow [54, 55, 1] to generate the ground truth boundaries and supervise our shape stream.”
BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations (2022, background): “Reducing labeling costs can be achieved in a variety of ways, such as interactive humanin-the-loop annotation [2, 50, 53, 64, 73], nearest-neighbor mask transfer [26, 51, 63, 70], or using weak supervision in the form of cheaper labels such as object boxes [7, 43, 45], scribbles [48, 69] or very coa”
Deep Interactive Thin Object Selection (2021, background): “Fortunately, this issue has been extensively studied in edge detection literature [7, 1].”
Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, background): “most applications, including segmentation (experiments in Section V-C; [103]; [104]; [105]), lesion detection (experiments in Section V-A), and classification ([106]) [107] - image classification [76] - image classification [108] - image classification [109] - image classification [110] - boundary s”
Semantic Edge Detection with Diverse Deep Supervision (2018, methodology): “STEAL and Gated-SCNN achieve similar performance, and both are much worse than DDS-R and DDS-U.”
Deepstrip: High-Resolution Boundary Refinement (2020, methodology): “Multiple attempts have been made to improve boundary quality through extracting better features [41, 43, 26, 2, 14].”
Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey (2019, background): “ion [35]–[38]. Furthermore, beside image classiﬁcation, there are more ﬁelds where dealing with mislabeled instances is important, such as generative networks [169], [170], semantic segmentation [28]–[30], sound classiﬁcation [171] and more. All these factors make dealing with label noise an importa”
RINDNet: Edge Detection for Discontinuity in Reflectance, Illumination, Normal and Depth (2021, background): “Recent works [1,47,49,50] pay more attention to special types of edges.”
SegFix: Model-Agnostic Boundary Refinement for Segmentation (2020, methodology): “Recent work [1,14,53,31] extended the conventional level-set scheme to deep network for regularizing the boundaries of predicted segmentation map.”
PolyTransform: Deep Polygon Transformer for Instance Segmentation (2019, methodology): “[1] develop an approach that can be used to reﬁne noisy annotations by jointly reasoning about the object boundaries with a CNN and a level set formulation.”