Learning to Segment Object Candidates

By Pedro H. O. Pinheiro, Ronan Collobert, Piotr Doll'ar. 2015

817 citations recorded by the source corpus.

Recent object detection systems rely on two critical steps: (1) a set of object proposals is predicted as efficiently as possible, and (2) this set of candidate proposals is then passed to an object classifier. Such approaches have been shown they can be fast, while achieving the state of the art in detection performance. In this paper, we propose a new way to generate object proposals, introducing an approach based on a discriminative convolutional network. Our model is trained jointly with two objectives: given an image patch, the first part of the system outputs a class-agnostic segmentation mask, while the second part of the system outputs the likelihood of the patch being centered on a full object. At test time, the model is efficiently applied on the whole test image and generates a set of segmentation masks, each of them being assigned with a corresponding object likelihood score. We show that our model yields significant improvements over state-of-the-art object proposal algorithms. In particular, compared to previous approaches, our model obtains substantially higher object recall using fewer proposals. We also show that our model is able to generalize to unseen categories it has not seen during training. Unlike all previous approaches for generating object masks, we do not rely on edges, superpixels, or any other form of low-level segmentation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015, methodology): “Concurrent with our work, theDeepMaskmethod [28] is developed for learning segmen-”
Mask R-CNN (2017, methodology): “Unlike previous methods that resort to fc layers for mask prediction [32, 33, 10], our fully convolutional representation requires fewer parameters, and is more accurate as demonstrated by experiments.”
Learning Visual Features from Large Weakly Supervised Data (2015, background): “…studies have shown that using visual features extracted from convolutional networks trained on large object recognition datasets [26, 43, 46] can lead to state-of-the-art results on many vision problems including ﬁne-grained classiﬁcation [21, 39], object detection [15], and segmentation [36].”
Learning Features by Watching Objects Move (2016, methodology): “Note that a similar training setup was used for recent segmentation proposal methods [37, 38].”
Feature Pyramid Networks for Object Detection (2016, methodology): “Existing mask proposal methods [27, 28, 4] are based on densely sampled image pyramids ( e.g ., scaled by 2 {− 2:0 .”
Panoptic Segmentation (2018, background): “By framing this task and providing a high-quality dataset, COCO helped define a new and exciting research direction and led to many recent breakthroughs in instance segmentation [36, 24, 14].”
Fully Convolutional Instance-Aware Semantic Segmentation (2016, methodology): “Recently, the task is achieved by deep learning approaches, such as DeepMask [32] and SharpMask [33].”
Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, background): “Another related approach [11, 49] is to use segment-based region proposals [9, 35, 36] instead of box-based proposals.”
Beyond Skip Connections: Top-Down Modulation for Object Detection (2016, methodology): “After the resurgence of ConvNets for image classification [8, 29], they have been successfully adopted for a variety of computer vision tasks such as object detection [16, 17, 41, 48], semantic segmentation [3, 6, 34, 35], instance segmentation [21, 22, 38], pose estimation [49, 50], depth estimatio”
Shape-aware Instance Segmentation (2016, methodology): “In particular, we report the results of MCG [1], Deep-Mask [23] and Sharp-Mask [24] using the publicly available pre-computed segmentation proposals.”
Deep Watershed Transform for Instance Segmentation (2016, methodology): “Current approaches generally use complex pipelines to handle instance extraction involving object proposals [20, 22, 7], conditional random fields (CRF) [32, 33], large recurrent neural networks (RNN) [24, 23, 2], or template matching [28].”
InstanceCut: From Edges to Instances with MultiCut (2016, methodology): “Several methods produce proposals for instance segmentations and combine them, based on learned scores [33, 41, 42] or generate parts of instances and then combine them [14, 38].”
Aggregated Residual Transformations for Deep Neural Networks (2016, background): “The robustness of VGGnets and ResNets has been proven by various visual recognition tasks [7, 10, 9, 28, 31, 14] and by non-visual tasks involving speech [42, 30] and language [4, 41, 20].”
Instance-Sensitive Fully Convolutional Networks (2016, methodology): “We use the deﬁnition of positive/negative samples in [8], and the 256 sampled windows have a positive/negative sampling ratio of 1:1 [19].”
Learning to Refine Object Segments (2016, methodology): “Simply upgrading the 11-layer VGG-A model [26] used in [22] to the 16-layer VGG-D model can double run time.”
Instance-Aware Semantic Segmentation via Multi-task Network Cascades (2015, methodology): “DeepMask [25] is recently developed for learning segmentation candidates using convolutional networks, taking over 1 second per image.”
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks (2015, methodology): “For all experiments, we are using selective search [40] for object proposals, which performs very poorly on small objects in COCO with an average recall under 10% [29].”
What Makes for Effective Detection Proposals? (2015, background): “Indeed, there has been some very recent work in this space [60], [61], [62] that shows promising”
Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, methodology): “Open-world setup [5, 43] has been recently (re)-introduced into various problems in computer vision such as recognition [30, 38], tracking [36], detection [25, 27, 53], and segmentation [54].”
EDTER: Edge Detection with Transformer (2022, background): “Edge detection is one of the most fundamental problems in computer vision and has a wide variety of applications, such as image segmentation [8, 23, 39, 44, 45, 47], object detection [23], and video object segmentation [5, 57, 59].”