Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation

By Jordi Pont-Tuset, Pablo Arbeláez, Jonathan T. Barron, Ferran Marqués, Jitendra Malik. 2015

601 citations recorded by the source corpus.

We propose a unified approach for bottom-up hierarchical image segmentation and object proposal generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object proposals by exploring efficiently their combinatorial space. We also present Single-scale Combinatorial Grouping (SCG), a faster version of MCG that produces competitive proposals in under five seconds per image. We conduct an extensive and comprehensive empirical validation on the BSDS500, SegVOC12, SBD, and COCO datasets, showing that MCG produces state-of-the-art contours, hierarchical regions, and object proposals.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks (2015, methodology): “Legend: R: include 2x stacked 4-dir IRNN (context features), W: two rounds of bounding box regression and weighted voting [8], D: remove all dropout, +S: train with segmentation labels, SS: SelectiveSearch [40], MCG: MCG proposals [30], RPN: region proposal net [32].”
Learning to Segment Object Candidates (2015, methodology): “These approaches usually fall into three categories: (1) objectness scoring [1, 34], in which proposals are extracted by measuring the objectness score of bounding boxes, (2) seed segmentation [14, 16, 17], where models start with multiple seed regions and generate separate foreground-background seg”
Learning Open-World Object Proposals without Learning to Classify (2021, background): “In early works, the emphasis was on category-independent object proposals [21], [3], [22], [7], where the goal is to identify instances of all objects in the image irrespective of their category.”
Deep Extreme Cut: From Extreme Points to Object Segmentation (2017, methodology): “Popular grouping methods provide instance segmentation in the form of automatically segmented object proposals [14, 30].”
Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources (2015, methodology): “Then, we use Multiscale Combinatorial Grouping (MCG) [22] for the proposal generation.”
Proposal-Free Network for Instance-Level Object Segmentation (2015, neutral): “In general, these previous methods take complicated preprocessing such as bottom-up region proposal generation [25] [36] [41] [24] or post-processing such as graphical inference as the requisite.”
What Value Do Explicit High Level Concepts Have in Vision to Language Problems? (2015, methodology): “We use Multiscale Combinatorial Grouping (MCG) [42] for the proposal generation.”
Visual Semantic Role Labeling (2015, methodology): “We use the precomputed MCG bounding boxes from [27].”
Recurrent Pixel Embedding for Instance Grouping (2017, neutral): “SCG [71] MCG [71] COB [64] inst-DML [23] Fig.”
One-Shot Video Object Segmentation (2016, methodology): “The duality between multiscale contours and hierarchical segmentation [1, 40] was further studied by Maninis et al.”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, methodology): “We use GrabCut [48] (a classic foreground segmentation technique given a bounding-box prior) and MCG [50] (a segment-proposal algorithm) to obtain a foreground mask ).”
Learning Video Object Segmentation from Static Images (2016, methodology): “Given an estimate of the object location and size, bottom-up segment proposals [38] or GrabCut [42] variants can be used as shape guesses.”
PIXOR: Real-time 3D Object Detection from Point Clouds (2018, methodology): “Since the introduction of class-agnostic object proposals [36, 26], proposal based approaches populate, with Region-CNN (RCNN) [12] and its faster versions [11, 4] being the most seminal work.”
Exploiting Saliency for Object Segmentation from Image Level Labels (2017, methodology): “3 validation set results, however we do not consider these results comparable since they use the MCG scores [35], which are trained on the ground truth Pascal segments.”
Weakly Supervised Instance Segmentation Using Class Peak Response (2018, methodology): “Multi-scale Combinatorial Grouping (MCG) [27] uses low-level contour information, e.g ., Structured Edge [8] or Ultrametric Contour Map [20], to extract object proposals, which contain ﬁne-detailed object boundaries that is valuable to instance segmentation.”
What leads to generalization of object proposals? (2020, background): “Object proposal generation models: There have been many seminal works on generating class-agnostic object proposals [36,38,25,14].”
Simple Does It: Weakly Supervised Instance and Semantic Segmentation (2016, methodology): “We use the state of the art proposals from MCG [34].”
Semantic Image Interpretation - Integration of Numerical Data and Logical Knowledge for Cognitive Vision (2018, unknown): “Most of the object proposal works can be divided in two groups: (i) those based on grouping super-pixels (for example, Selective Search [106], Constrained Parametric Min-Cuts (CPMC) [13], Multiscale Combinatorial Grouping (MCG) [76]) and (ii) those based on sliding windows (for example, objectness i”