High-for-Low and Low-for-High: Efficient Boundary Detection from Deep Object Features and Its Applications to High-Level Vision

By Gedas Bertasius, Jianbo Shi, Lorenzo Torresani. 2015

187 citations recorded by the source corpus.

Most of the current boundary detection systems rely exclusively on low-level features, such as color and texture. However, perception studies suggest that humans employ object-level reasoning when judging if a particular pixel is a boundary. Inspired by this observation, in this work we show how to predict boundaries by exploiting object-level features from a pretrained object-classification network. Our method can be viewed as a "High-for-Low" approach where high-level object features inform the low-level boundary detection process. Our model achieves state-of-the-art performance on an established boundary detection benchmark and it is efficient to run. Additionally, we show that due to the semantic nature of our boundaries we can use them to aid a number of high-level vision tasks. We demonstrate that using our boundaries we improve the performance of state-of-the-art methods on the problems of semantic boundary labeling, semantic segmentation and object proposal generation. We can view this process as a "Low-for-High'" scheme, where low-level boundaries aid high-level vision tasks. Thus, our contributions include a boundary detection system that is accurate, efficient, generalizes well to multiple datasets, and is also shown to improve existing state-of-the-art high-level vision methods on three distinct tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

Citations in the corpus

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, methodology): “In a different direction, [63] replace the bilateral filtering module used in mean field inference with a faster domain transform module [67], improving the speed and lowering the memory requirements of the overall system, while [18], [68] combine semantic segmentation with edge detection.”
UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory (2016, neutral): “0 High-for-Low [5] 47.”
Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform (2015, background): “Our work is most related to the ones by [44, 3, 24].”
Unsupervised Learning of Edges (2015, methodology): “Since the work of [10], which formulated edge detection as a binary classification problem, progressively more powerful learning paradigms have been employed, including multi-class classification [23], feature learning [30], regression [35], structured prediction [11], and deep learning [20, 5, 44].”
Understanding Convolution for Semantic Segmentation (2017, methodology): “…fully connected pairwise CRFs [22] as a post-processing step [7, 8], integrating CRFs into the network by approximating its mean-ﬁeld inference steps [43, 27, 24, 32] to enable end-to-end training, and incorporating additional information into CRFs such as edges [21, 4] and object detections [2].”
One-Shot Video Object Segmentation (2016, methodology): “Image segmentation has also been taken over by CNNs recently [29, 22, 53, 3, 4], with deep architectures pre-trained on the weakly related task of image classification on ImageNet [44].”
DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation (2016, background): “Recently, deep neural networks are driving advances in image recognition related tasks in computer vision [21, 9, 7, 27, 29, 3] and medical image computing [10, 30, 31, 6, 12].”
DCAN: Deep contour‐aware networks for object instance segmentation from histology images (2017, methodology): “It is worth noting that contour detection and object segmentation are two complementary tasks and previous studies showed preliminarily promising results by exploring semantic contours on PASCAL dataset ( Bertasius et al., 2015a; Chen et al., 2015c ).”
Semantic Segmentation with Boundary Neural Fields (2015, methodology): “We note that we also compared the results of our inference technique using other edge detectors, notably UCM [1] and HFL [4].”
Self-Supervised Visual Representation Learning from Hierarchical Grouping (2020, unknown): “Though deep neural networks are now the dominant tool for contour detection [41, 4, 46, 25, 47], the best prior approaches [2, 3, 11] deliver somewhat respectable results.”
SegFix: Model-Agnostic Boundary Refinement for Segmentation (2020, methodology): “We follow the contour detection methods [4,2,20] and simply use a convolutional network to predict a binary mask indicating the boundary pixels.”
First Person Action-Object Detection with EgoNet (2016, background): “We note that the recent introduction of the fully convolutional networks (FCNs) [35] has led to remarkable results in a variety of structured prediction tasks such as edge detection [3, 51, 25] and semantic image segmentation [8, 55, 34, 32, 38, 20, 9, 7].”
Pixel Difference Networks for Efficient Edge Detection (2021, methodology): “Canny [5] and SE [9] edge detectors are usually used to extract candidate contour points before applying the CNN model for contour/noncontour prediction [2, 3].”
Learning Deep Representation for Imbalanced Classification (2016, methodology): “Recent HFL [5], HED [45] methods utilize large VG- Figure 5.”
Object Contour Detection with a Fully Convolutional Encoder-Decoder Network (2016, background): “[7] improve semantic contour detection with convolutional features and shows its application to semantic segmentation.”