ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation

By Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He. 2016

1,133 citations recorded by the source corpus.

Large-scale data is of crucial importance for learning semantic segmentation models, but annotating per-pixel masks is a tedious and inefficient procedure. We note that for the topic of interactive image segmentation, scribbles are very widely used in academic research and commercial software, and are recognized as one of the most userfriendly ways of interacting. In this paper, we propose to use scribbles to annotate images, and develop an algorithm to train convolutional networks for semantic segmentation supervised by scribbles. Our algorithm is based on a graphical model that jointly propagates information from scribbles to unmarked pixels and learns network parameters. We present competitive object semantic segmentation results on the PASCAL VOC dataset by using scribbles as annotations. Scribbles are also favored for annotating stuff (e.g., water, sky, grass) that has no well-defined shape, and our method shows excellent results on the PASCALCONTEXT dataset thanks to extra inexpensive scribble annotations. Our scribble annotations on PASCAL VOC are available at http://research.microsoft.com/en-us/um/ people/jifdai/downloads/scribble_sup.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “Scribble-based methods often use heuristic and complicated procedures for simulating user input, which makes it hard to fairly evaluate them [5, 6, 7].”
Iteratively Trained Interactive Segmentation (2018, methodology): “[18] use scribbles as a form of supervision for a Fully Convolutional Network (FCN) to segment images.”
Deep Extreme Cut: From Extreme Points to Object Segmentation (2017, methodology): “Image-level labels [26], noisy web labels [1, 16] and scribble-level labels [20] are some of the supervisory signal that have been used to guide segmentation methods.”
Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “To address this issue, Weakly Supervised Semantic Segmentation (WSSS) methods aim to obtain effective segmentation models using cheaper supervisions such as bounding boxes [18, 32, 49], scribbles [39, 60], points [6, 53], and image-level labels [31, 34, 35, 59].”
What's the Point: Semantic Segmentation with Point Supervision (2015, methodology): “semantic segmentation, recent works have focused on training models in a weakly- or semi-supervised setting. Many forms of supervision have been explored, such as eye tracks [26], free-form squiggles [17,18], noisy web tags [27], size constraints on objects [6] or heterogeneous annotations [33]. Com”
InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting (2019, background): “These researches mainly follow two lines: i) transform annotations from other domain to object masks [12, 30] or ii) utilize data from other domain as extra regularization term [21, 4].”
Guide Me: Interacting with Deep Networks (2018, methodology): “Nonetheless, some methods towards interactive deep learning have been proposed, such as weakly-supervised semantic segmentation from scribbles [38], user-provided clicks and Euclidean distance maps [61] or bounding boxes used as region initialization [17, 53].”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, background): “Subsequently, many approaches have achieved high accuracy using only image-level labels [9, 10, 40, 41], bounding boxes [42, 11, 12], scribbles [20] and points [13].”
Learning What Not to Segment: A New Perspective on Few-Shot Segmentation (2022, background): “However, collecting sufficient labeled data is notoriously time-consuming and labor-intensive, especially for dense prediction tasks, such as instance segmentation [2,3,15,21,59] and semantic segmentation [1,25,35,40,45].”
An Overview of Deep Semi-Supervised Learning (2020, background): “For example, in weakly-supervised semantic segmentation, pixel-level labels, which are harder and more expensive to acquire, are substituted for inexact annotations, e . g ., image labels [159, 184, 161, 97, 94], points [9], scribbles [100] and bounding boxes [144, 31].”
Weakly Supervised Instance Segmentation Using Class Peak Response (2018, neutral): “ning phase. Given the inefﬁciency of pixellevel annotating, previous efforts have explored various alternative weak annotations, e.g., points on instances [1], object bounding boxes [5,22], scribbles [17,42], and human selected foreground [34]. Although effective, these Bird Sheep Image c-MWP attent”
Simple Does It: Weakly Supervised Instance and Semantic Segmentation (2016, background): “In order to keep annotation cost low, recent work has explored different forms of supervision for semantic labelling: image labels [29, 28, 27, 30, 42], points [3], scribbles [44, 23], and bounding boxes [8, 27].”
Semi-Supervised Semantic Segmentation With High- and Low-Level Consistency (2019, background): “Attempting to alleviate the problem, several methods exploit weaker forms of supervision: image-level labels [1], [32], [38], bounding boxes [7], [29], or scribbles [20], [36].”
Webly Supervised Semantic Segmentation (2017, methodology): “Finally {R k } is adopted again to train an end-to-end semantic segmentation network. points, scribbles, or bounding boxes, have also been employed to supervise semantic segmentation [9] [10] [11] [12].”
Semantic Projection Network for Zero- and Few-Label Semantic Segmentation (2019, methodology): “As weak supervision, prior works use image-level annotation [34, 32, 31], point [5], scribble [25] and bounding box [20] annotations.”
Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation (2018, methodology): “Supervision: Scribbles Scribblesup CVPR2016 [17] 10K 63.”
Self-Erasing Network for Integral Object Attention (2018, background): “Besides some works relying on relatively strong supervisions, such as scribble [19], points [1], and bounding boxes [27], most”