Simple Does It: Weakly Supervised Instance and Semantic Segmentation

By Anna Khoreva, Rodrigo Benenson, Jan Hendrik Hosang, Matthias Hein, Bernt Schiele. 2016

795 citations recorded by the source corpus.

Semantic labelling and instance segmentation are two tasks that require particularly costly annotations. Starting from weak supervision in the form of bounding box detection annotations, we propose a new approach that does not require modification of the segmentation training procedure. We show that when carefully designing the input labels from given bounding boxes, even a single round of training is enough to improve over previously reported weakly supervised results. Overall, our weak supervision approach reaches ~95% of the quality of the fully supervised model, both for semantic labelling and instance segmentation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Extreme Cut: From Extreme Points to Object Segmentation (2017, methodology): “Also related to our approach, [7, 17] train semantic segmentation methods from box supervision.”
Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “To address this issue, Weakly Supervised Semantic Segmentation (WSSS) methods aim to obtain effective segmentation models using cheaper supervisions such as bounding boxes [18, 32, 49], scribbles [39, 60], points [6, 53], and image-level labels [31, 34, 35, 59].”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, methodology): “[37] propose a weakly supervised instance segmentation model given only bounding box annotations.”
Learning to Adapt Structured Output Space for Semantic Segmentation (2018, background): “As a result, weakly and semi-supervised approaches [5, 14, 17, 28, 29] are proposed in recent years to reduce the heavy labeling cost of collecting segmentation ground truths.”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, methodology): “Following recent work [42, 9, 13, 41], we use our weak supervision and image priors to approximate the ground-truth for a subset Ω ′ ⊂ Ω of the pixels in the image.”
Bootstrapping the Performance of Webly Supervised Semantic Segmentation (2018, methodology): “It is similar to [12], but we use the mask as prior knowledge to indicate the foreground and background instead of the bound box.”
Pseudo-mask Matters in Weakly-supervised Semantic Segmentation (2021, methodology): “To reduce the annotation burden, weaklysupervised semantic segmentation approaches have been proposed using scribble annotations [23, 32], bounding boxes [36, 9, 16] , points [3] or image-level labels [17, 27, 28, 2, 33].”
Adversarial Learning for Semi-supervised Semantic Segmentation (2018, background): “In addition to image-level supervisions, the segmentation network can also be trained with bounding boxes [6, 19], point supervision [2], or web videos [16].”
Learning Integral Objects With Intra-Class Discriminator for Weakly-Supervised Semantic Segmentation (2020, background): “…estimating pseudo-masks for the WSSS. of pixel-level annotations, researchers proposed weakly-supervised semantic segmentation (WSSS), which only adopts easily obtained coarse labels for training, e.g., image-level labels [20, 40, 1, 38, 37, 15], scribbles [24, 36], and bounding boxes [7, 19, 34].”
BoxInst: High-Performance Instance Segmentation with Box Annotations (2020, methodology): “SDI [17] might be the first instance segmentation framework with box annotations.”
On Regularized Losses for Weakly-supervised CNN Segmentation (2018, methodology): “The main stream of weakly-supervised methods generate segmentation proposals and train with such ”fake” ground truth [17,25,26,13,14,27].”
Learning Pixel-Level Semantic Affinity with Image-Level Supervision for Weakly Supervised Semantic Segmentation (2018, background): “Successful examples of weak supervision for semantic segmentation include bounding box [6, 12, 28], scribble [18, 36], point [1], and so on.”
Semi-Supervised Semantic Segmentation With High- and Low-Level Consistency (2019, methodology): “semi-supervised schemes [16], [29], [39] additionally use a few pixel-wise segmentation labels.”
Weakly Supervised Instance Segmentation Using Class Peak Response (2018, background): “[13] propose to obtain pseudo ground truth masks from bounding box supervision to alleviate labeling cost.”
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision (2021, neutral): “Method Backbone Arch AP25 AP50 AP70 AP75 SDI [46] VGG-16 DeepLabv2 - 44.”
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation (2020, methodology): “some human-annotated images and abundant pseudolabeled [42,72,34,3] video sequences. Semi-supervised learning has been widely applied to several computer vision tasks, including semantic segmentation [57,19,59,31,80,35,81,73,96], object detection [68,76,64], instance segmentation [35,60], panoptic s”
Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features (2018, background): “While fully-supervised methods require a large number of pixel-wise annotation which is very expensive, recent advances have exploited semantic segmentation with weak supervision, including bounding box [4, 19, 12], scribble [15] and image-level labels [21, 22, 25, 19, 31, 13, 23, 30].”
Learning to Segment Every Thing (2017, methodology): “In addition, we also compare with the following baseline approaches: a) unsupervised mask prediction using GrabCut [35] foreground segmentation over the Faster R-CNN detected object boxes (Faster R-CNN tested w/ GrabCut) and b) weakly supervised instance segmentation similar to [18], which trains an”
Puzzle-CAM: Improved Localization Via Matching Partial And Full Features (2021, methodology): “Unlike FSSS, which requires pixel-wise labels for an image, WSSS employs lower level labeling, such as bounding boxes [9], scribbles [10], and image-level classification labels [4, 6].”
Semi-supervised Segmentation Based on Error-Correcting Supervision (2020, background): “Weakly-supervised segmentation models generate dense classiﬁcation maps despite only image level [32, 35] or bounding box annotations [17] being present.”