Pointly-Supervised Instance Segmentation

By Bowen Cheng, Omkar M. Parkhi, Alexander Kirillov. 2021

149 citations recorded by the source corpus.

We propose an embarrassingly simple point annotation scheme to collect weak supervision for instance segmentation. In addition to bounding boxes, we collect binary labels for a set of points uniformly sampled inside each bounding box. We show that the existing instance segmentation models developed for full mask supervision can be seamlessly trained with point-based supervision collected via our scheme. Remarkably, Mask R-CNN trained on COCO, PASCAL VOC, Cityscapes, and LVIS with only 10 annotated random points per object achieves 94%−98% of its fully-supervised performance, setting a strong baseline for weakly-supervised instance segmentation. The new point annotation scheme is approximately 5 times faster than annotating full object masks, making high-quality instance segmentation more accessible in practice. Inspired by the point-based annotation form, we propose a modification to PointRend instance segmentation module. For each object, the new architecture, called Implicit PointRend, generates parameters for a function that makes the final point-level mask prediction. Implicit PointRend is more straightforward and uses a single point-level mask loss. Our experiments show that the new module is more suitable for the point-based supervision.11Project page: https://bowenc0221.github.io/point-sup

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

In Evaluation & Benchmarks

Citations in the corpus

Masked-attention Mask Transformer for Universal Image Segmentation (2021, methodology): “Motivated by PointRend [30] and Implicit PointRend [13], which show a segmentation model can be trained with its mask loss calculated on K randomly sampled points instead of the whole mask, we calculate the mask loss with sampled points in both the matching and the ﬁnal loss calculation.”
Incremental Learning in Semantic Segmentation from Image Labels (2021, background): “To reduce the annotation cost, different types of weak supervision have been proposed: bounding boxes [18, 32], scribbles [39, 62], points [16], and image-level labels [34, 50, 52].”
End-to-End Instance Edge Detection (2022, background): “Finally, related to our point supervised edge detection loss, [31,11] also perform point supervised learning, however, they sample points on the model’s predicted feature maps instead of using ground truth annotations.”