Pushing the Boundaries of Boundary Detection using Deep Learning

By Iasonas Kokkinos. 2015

245 citations recorded by the source corpus.

In this work we show that adapting Deep Convolutional Neural Network training to the task of boundary detection can result in substantial improvements over the current state-of-the-art in boundary detection. Our contributions consist firstly in combining a careful design of the loss for boundary detection training, a multi-resolution architecture and training with external data to improve the detection accuracy of the current state of the art. When measured on the standard Berkeley Segmentation Dataset, we improve theoptimal dataset scale F-measure from 0.780 to 0.808 - while human performance is at 0.803. We further improve performance to 0.813 by combining deep learning with grouping, integrating the Normalized Cuts technique within a deep network. We also examine the potential of our boundary detector in conjunction with the task of semantic segmentation and demonstrate clear improvements over state-of-the-art systems. Our detector is fully integrated in the popular Caffe framework and processes a 320x420 image in less than a second.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, methodology): “In a different direction, [63] replace the bilateral filtering module used in mean field inference with a faster domain transform module [67], improving the speed and lowering the memory requirements of the overall system, while [18], [68] combine semantic segmentation with edge detection.”
EDTER: Edge Detection with Transformer (2022, methodology): “Recently, convolutional neural networks (CNNs) have been successfully introduced the edge detection study [3, 4, 11, 12, 26, 29, 40, 46, 48, 52, 66].”
Holistically-Nested Edge Detection (2015, background): “…there have emerged a number of properties that a researcher in the ﬁeld might generally agree are key and that are likely to play a role in a successful system: (1) carefully designed and/or learned features [26, 5], (2) multi-scale response fusion [38, 30, 28], (3) engagement of different…”
Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, background): “Due to the class imbalance between the edge and non-edge pixels, edge pixels are penalized with a weight 0.95, while non-edge pixels with a scale of 0.05, accommodating [24,38].”
Higher Order Conditional Random Fields in Deep Neural Networks (2015, background): “…improves the Interior IoU over our baseline (only pairwise potentials [53]) as the object Ours 77.9 DPN [34] 77.5 Centrale Super Boundaries [22] 75.7 Dilated Convolutions [52] 75.3 BoxSup [6] 75.2 DeepLab Attention [5] 75.1 CRF-RNN (baseline) [53] 74.7 DeepLab WSSL [37] 73.9 DeepLab [4]…”
Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform (2015, background): “Our work is most related to the ones by [44, 3, 24].”
Fast, Exact and Multi-scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs (2016, background): “As in [27], we extend it to get a multi-resolution network, which operates at three resolutions with tied weights.”
Recurrent Pixel Embedding for Instance Grouping (2017, background): “Note that it is possible to surpass human performance with several sophisticated techniques [44], we don’t pursue this as it is out the scope of this paper.”
Laplacian Reconstruction and Refinement for Semantic Segmentation (2016, methodology): “One approach is to combine CNN predictions with a conditional random ﬁeld (CRF) that utilizes bottom-up superpixels [8,9] or generic boundary detection [10,11] to precisely localize segment boundaries.”
Understanding Convolution for Semantic Segmentation (2017, methodology): “This includes applying fully connected pairwise CRFs [16] as a post-processing step [3], integrating CRFs into the network by approximating its mean-field inference steps [31, 20, 18] to enable end-to-end training, and incorporating additional information into CRFs such as edges [15] and object dete”
One-Shot Video Object Segmentation (2016, methodology): “Equation 1 allows training for imbalanced binary tasks [22, 53, 29, 30].”
Image-Based Localization Using LSTMs for Structured Feature Correlation (2016, methodology): “This has been successfuly applied to object detection [13], object segmentation [26,36], semantic segmentation [19,37], depth and normal estimation [29], to name a few.”
Video Object Segmentation without Temporal Information (2017, methodology): “Equation (1) allows training for imbalanced binary tasks [31], [42], [43], [72].”
Deep Structural Contour Detection (2020, methodology): “….757 .776 1 / 30 † DeepEdge [3] .753 .772 1 / 1000 † HFL [4] .767 .788 5 / 6 † HED [46] .788 .808 30 † CEDN [47] .788 .804 10 MIL+G-DSN+MS+NCuts [19] .813 .831 1 RCF-VOC-aug [27] .806 .823 30 † RCF-MS-VOC-aug [27] .811 .830 10 † RCF-ResNet101-MS-VOC-aug [27] .819 .836 10 † CED [44] .794 .811 30…”
Self-Supervised Visual Representation Learning from Hierarchical Grouping (2020, background): “Though deep neural networks are now the dominant tool for contour detection [41, 4, 46, 25, 47], the best prior approaches [2, 3, 11] deliver somewhat respectable results.”
MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning (2020, methodology): “As in prior work [19, 29, 30], the edge detection task is trained with a positively weighted w pos = 0 .”