Multi-scale Patch Aggregation (MPA) for Simultaneous Detection and Segmentation

By Shu Liu, Xiaojuan Qi, Jianping Shi, Hong Zhang, Jiaya Jia. 2016

81 citations recorded by the source corpus.

Aiming at simultaneous detection and segmentation (SD-S), we propose a proposal-free framework, which detect and segment object instances via mid-level patches. We design a unified trainable network on patches, which is followed by a fast and effective patch aggregation algorithm to infer object instances. Our method benefits from end-to-end training. Without object proposal generation, computation time can also be reduced. In experiments, our method yields results 62.1% and 61.8% in terms of mAPr on VOC2012 segmentation val and VOC2012 SDS val, which are state-of-the-art at the time of submission. We also report results on Microsoft COCO test-std/test-dev dataset in this paper.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recurrent Neural Networks for Semantic Instance Segmentation (2017, methodology): “Following standard practices in [3,22,32], we train with the additional annotations from [33] and evaluate on the original validation set, composed of 1,449 images.”
Fully Convolutional Instance-Aware Semantic Segmentation (2016, background): “There are some endeavors [22, 26] trying to extend FCNs for instance-aware semantic segmentation, by grouping/clustering the FCN’s output.”
Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, methodology): “At an IoU threshold of 0.9, our improvement over the previous state-of-the-art (MPA [37]) is 6.6%, which is a relative improvement of 36%.”
InstanceCut: From Edges to Instances with MultiCut (2016, methodology): “ these proposals [13,21,15,11]. Several methods produce proposals for instance segmentations and combine them, based on learned scores [33, 41,42] or generate parts of instances and then combine them [14,38]. Although the proposal-based methods show stateof-the-art performance on important challenge”
Path Aggregation Network for Instance Segmentation (2018, methodology): “Motivation Fully-connected layers, or MLP, were widely used in mask prediction in instance segmentation [10, 41, 34] and mask proposal generation [48, 49].”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, background): “[50] segment objects in multiscale patches and aggregate the results.”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, background): “Most top-performing algorithms modify object detection networks to output a ranked list of segments instead of boxes [22–26 , 37].”
Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers (2021, unknown): “Previous approaches resolve the mask conflict between neighboring objects through non-maximum suppression or additional post-processing [43, 14, 34, 30, 20].”
An End-To-End Network for Panoptic Segmentation (2019, methodology): “The proposal-based approaches [8, 14, 24, 25, 28, 29, 33] first generate the object detection bounding boxes and then perform mask prediction on each box for instance segmentation.”
BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks (2017, methodology): “We compare BiSeg with recent methods [2, 7, 8, 9, 16, 17, 19, 20, 23], and variants of our own method with different settings.”
CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos (2017, methodology): “The fully convolutional architecture and learnable upsampling method are efficient and effective, and thus inspired many extensions [37, 19, 33, 4, 32, 79, 5, 6, 74].”
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes (2017, background): “The works in [26, 33] try to adapt FCNs to instancespecific semantic segmentation by applying some sort of clustering to the FCNs’ representations.”
Semantic Segmentation With Context Encoding and Multi-Path Decoding (2020, result): “[47] generated multi-scale patches and aggregated the results.”
SGN: Sequential Grouping Networks for Instance Segmentation (2017, background): “Note that most instance segmentation algorithms cannot tackle the fragmentation problem [3, 26, 18, 41, 40].”
Survey on semantic segmentation using deep learning techniques (2019, background): “Liu et al. [110] proposed the strategy named Multi-scale Patch Aggregation (MPA).”
Associative Embedding: End-to-End Learning for Joint Detection and Grouping (2016, methodology): “g Figure 11. Example instance predictions produced by our system on the PASCAL VOC 2012 validation set. mAP @ IOU=0.5 mAP @ IOU=0.7 SDS [12] 49.7 25.3 Hypercolumn [13] 60.0 40.4 CFM [4] 60.7 39.6 MPA [20] 61.8 – MNC [5] 63.5 41.5 Our method 35.1 26.0 Table 2. Semantic instance segmentation results o”