BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

By Hao Chen, Kunyang Sun, Zhi Tian, Chunhua Shen, Yongming Huang, Youliang Yan. 2020

574 citations recorded by the source corpus.

Instance segmentation is one of the fundamental vision tasks. Recently, fully convolutional instance segmentation methods have drawn much attention as they are often simpler and more efficient than two-stage approaches like Mask R-CNN. To date, almost all such approaches fall behind the two-stage Mask R-CNN method in mask precision when models have similar computation complexity, leaving great room for improvement. In this work, we achieve improved mask prediction by effectively combining instance-level information with semantic information with lower-level fine-granularity. Our main contribution is a blender module which draws inspiration from both top-down and bottom-up instance segmentation approaches. The proposed BlendMask can effectively predict dense per-pixel position-sensitive instance features with very few channels, and learn attention maps for each instance with merely one convolution layer, thus being fast in inference. BlendMask can be easily incorporate with the state-of-the-art one-stage detection frameworks and outperforms Mask R-CNN under the same training schedule while being faster. A light-weight version of BlendMask achieves 36.0 mAP at 27 FPS evaluated on a single 1080Ti. Because of its simplicity and efficacy, we hope that our BlendMask could serve as a simple yet strong baseline for a wide range of instance-wise prediction tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

3D Instance Segmentation of MVS Buildings (2021, methodology): “Inspired by full convolutional instance-aware semantic segmentation (FCIS) [29] and You Only Look At Coefficients (YOLACT) [30], BlendMask uses a blender module to merge top-level coarse instance information with lower-level fine granularities [31].”
Class-Incremental Instance Segmentation via Multi-Teacher Networks (2021, background): “…development of deep learning, like other visual tasks (Wei et al. 2019; Yang et al. 2019; Han et al. 2020; Feng et al. 2020), current instance segmentation methods (He et al. 2017; Bolya et al. 2019; Chen et al. 2020) based on convolutional neural networks (CNNs) have achieved remarkable results.”
SOLQ: Segmenting Objects by Learning Queries (2021, unknown): “Top-down approaches [20, 2, 8, 21, 22, 23, 3, 5, 24] follow the detect-then-segment pipeline.”
Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation (2020, methodology): “In the experiments, CIoU loss and Cluster-NMS have been applied to several state-of-the-art instance segmentation (e.g., YOLACT [20] and BlendMask-RT [42]) and object detection (e.g., YOLO v3 [9], SSD [10] and Faster R-CNN [15]) models.”
A survey on instance segmentation: state of the art (2020, unknown): “5% 2019 BlendMask [135] 41.”
Interactive Object Segmentation With Inside-Outside Guidance (2022, unknown): “…witnessed a revolutionary advancement in semantic [9], [10], [11], [12], [13], [14], [15], [16], [17], [18] and instance segmentation [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29] for different domains, such as general scenes [3], [30], [31], autonomous driving [5], [32],…”
SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation (2021, methodology): “Inspired by [5, 38], we generalize the mask prediction by modeling spatial scales of objects with semantic information for lower-level granularity.”
Crossover Learning for Fast Online Video Instance Segmentation (2021, background): “Recently, instance segmentation methods based on one-stage frameworks without explicit feature alignment operation begin to emerge [5, 4, 8, 65, 58, 59].”
FCOS: A Simple and Strong Anchor-Free Object Detector (2020, methodology): “Thus, following BlendMask-RT [2], we remove P 6 and P 7 , further reducing the inference time.”
Interactive Object Segmentation With Inside-Outside Guidance (2020, background): “Over the past few years, we have witnessed a revolutionary advancement in semantic [44, 40, 68, 69, 10, 11, 12, 30, 15, 31] and instance segmentation [25, 36, 8, 60, 13, 65, 34, 4, 9, 43, 29] for different domains, such as general scenes [20, 41, 70], autonomous driving [17, 48, 21], aerial imagery ”
Unseen Object Instance Segmentation for Robotic Environments (2020, methodology): “Other methods examine dense sliding-window instance segmentation on 4D tensors [40], combining top-down and bottom-up methods via blending modules [41], and alternative mask representations such as”