Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks

By Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Pablo Arbeláez, Luc Van Gool. 2016

249 citations recorded by the source corpus.

We present Convolutional Oriented Boundaries (COB), which produces multiscale oriented contours and region hierarchies starting from generic image classification Convolutional Neural Networks (CNNs). COB is computationally efficient, because it requires a single CNN forward pass for multi-scale contour detection and it uses a novel sparse boundary representation for hierarchical segmentation; it gives a significant leap in performance over the state-of-the-art, and it generalizes very well to unseen categories and datasets. Particularly, we show that learning to estimate not only contour strength but also orientation provides more accurate results. We perform extensive experiments for low-level applications on BSDS, PASCAL Context, PASCAL Segmentation, and NYUD to evaluate boundary detection performance, showing that COB provides state-of-the-art contours and region hierarchies in all datasets. We also evaluate COB on high-level tasks when coupled with multiple pipelines for object proposals, semantic contours, semantic segmentation, and object detection on MS-COCO, SBD, and PASCAL; showing that COB also improves the results for all tasks.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Holistically-Nested Edge Detection (2015, background): “Recent work (Yang et al. 2016; Maninis et al. 2016) shows edge detection results on the original PASCAL VOC dataset.”
Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, neutral): “05, accommodating [24,38].”
Recurrent Pixel Embedding for Instance Grouping (2017, methodology): “SCG [71] MCG [71] COB [64] inst-DML [23] Fig.”
Learning Video Object Segmentation from Static Images (2016, methodology): “Given a video sequence, we compute the optical ﬂow using EpicFlow [37] with Flow Fields matches [2] and convolutional boundaries [27].”
MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning (2020, methodology): “ks [13]. All our experiments are conducted with pre-trained ImageNet weights. We use the L1 loss for depth estimation and the cross-entropy loss for semantic segmentation on NYUD-v2. As in prior work [18,28,29], the edge detection task is trained with a positively weighted w pos= 0:95 binary cross-e”
Semantic Edge Detection with Diverse Deep Supervision (2018, methodology): “We compare DDS-R/DDS-U on the SBD dataset with several state-of-the-art methods including InvDet [1], HFL-FC8 [4], HFL-CRF [4], BNF [54], WS [47], DilConv [46], DSN [2], COB [3], CASENet [2] and SEAL [23].”
Open-Vocabulary Image Segmentation (2021, methodology): “COB [31] improves the ef-ﬁciency and performance by leveraging deep nets.”
Bayesian Semantic Instance Segmentation in Open Set World (2018, methodology): “C Assume that we have a method (e.g., COB [12]) that is able to estimate a contour probability map from the image.”
Understanding Deep Learning Techniques for Image Segmentation (2019, background): “convolutional features are also very useful from that perspective [139].”
Bi-Directional Cascade Network for Perceptual Edge Detection (2019, unknown): “778 – COB-ResNet50[33] RGB-HHA .”
Weakly Supervised Instance Segmentation Using Class Peak Response (2018, methodology): “Multi-scale Combinatorial Grouping (MCG) [27] uses low-level contour information, e.g ., Structured Edge [8] or Ultrametric Contour Map [20], to extract object proposals, which contain ﬁne-detailed object boundaries that is valuable to instance segmentation.”
Piecewise Flat Embedding for Image Segmentation (2018, methodology): “It is splited into 4,998 training images, 2,607 validation images and 2,498 testing images in [31].”
Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Poorly-Drawn Sketches (2019, background): “Contours can be obtained by the boundary contour edge extractors like HED [45], COB [59], [60], RCF [61], [62], or similar to pix2pixHD [5], simpliﬁed from the face parsing semantic labels.”
Video Object Segmentation without Temporal Information (2017, methodology): “The advances observed in image recognition [22], [32], [62] have been beneficial to segmentation in many forms (semantic [40], [47], instance-level [7], [15], [52], biomedical [59], generic [41], etc.”
Pixel-Wise Attentional Gating for Scene Parsing (2018, methodology): “Following [56, 41, 31], we include four prediction branches at macro residual blocks (denoted by Res 2 , 3 , 4 , 5 ) and a fusion branch for training.”
Scaling Open-Vocabulary Image Segmentation with Image-Level Labels (2021, methodology): “These mid-level region representations are then used for semantic segmentation [33] and object detection [45].”
Attentive Single-Tasking of Multiple Tasks (2019, methodology): “As is common practice [63, 27, 36], the positive pixels are weighted more (0.”