Iteratively Trained Interactive Segmentation

By Sabarinath Mahadevan, Paul Voigtlaender, Bastian Leibe. 2018

134 citations recorded by the source corpus.

Deep learning requires large amounts of training data to be effective. For the task of object segmentation, manually labeling data is very expensive, and hence interactive methods are needed. Following recent approaches, we develop an interactive object segmentation system which uses user input in the form of clicks as the input to a convolutional network. While previous methods use heuristic click sampling strategies to emulate user clicks during training, we propose a new iterative training strategy. During training, we iteratively add clicks based on the errors of the currently predicted segmentation. We show that our iterative training strategy together with additional improvements to the network architecture results in improved results over the state-of-the-art.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Segment Anything (2023, methodology): “We adapt this method from interactive segmentation [109, 70], although unlike interactive segmentation whose aim is to eventually predict a valid mask after enough user input, our aim is to always predict a valid mask for any prompt even when the prompt is ambiguous.”
FocalClick: Towards Practical Interactive Image Segmentation (2022, methodology): “For the strategy of click simulation, we exploit iterative training [29] following RITM [36].”
Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “Following [12], we combine the random and iterative sampling strategies for training our iterative models.”
Getting to 99% Accuracy in Interactive Segmentation (2020, methodology): “To integrate the click maps into the network, most papers (iFCN [6], RIS-Net [13], ITIS [7], DEXTR [8]) adopt an early fusion scheme by simply joining the click maps as extra channels to the input image tensor.”
Iterative Interaction Training for Segmentation Editing Networks (2018, neutral): “In recent years, studies such as [1,9,19] proposed CNNs for interactive seg-mentations and showed better results compared to traditional methods.”
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation (2023, unknown): “We follow [22] to generate the random sampling and [23] to simulate the iterative sampling.”
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click (2020, background): “Many interactive segmentation methods have been proposed in the past decades using many diﬀerent interaction types such as bounding boxes [16, 29, 49, 57], contours [1,6,26,45], strokes [5,31,51,53] and clicks [25,28,34,40,41,58].”
Interactive Object Segmentation With Inside-Outside Guidance (2020, methodology): “Interactive Segmentation: Most interactive segmentation methods target at relieving human effort when labeling pixel-level annotations with the help of bounding box [66, 64, 52], clicks [67, 38, 37, 45] or contours [7, 1, 42] as guidance.”
Interactive Object Segmentation With Inside-Outside Guidance (2022, methodology): “Thus, interactive segmentation, which allows the human annotators to quickly extract the object-of-interest by providing some user inputs such as bounding boxes [2], [35], [36] or clicks [37], [38], [39], [40], appears to be an attractive and efﬁcient way to reduce the annotation effort.”
A Fully Convolutional Two-Stream Fusion Network for Interactive Image Segmentation (2018, background): “Essentially, all these networks [15, 17, 16, 18, 19, 21, 28] adopt early fusion structures.”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “For the global view, we adopt the iterative train-2639 ing strategy [33].”
Interactive Image Segmentation With First Click Attention (2020, methodology): “We also employ the same iterative training strategy [36, 37], which improves the effect of the ﬁnal model to a certain extent.”
Conditional Diffusion for Interactive Segmentation (2021, unknown): “In a standard pipeline, the first step is encoding clicks into distance maps [32, 27, 11, 31], Gaussian maps [20, 23, 21, 14], or super-pixels [22]; next, encoded maps is concatenated with the original image and fed into a segmentation network to make predictions.”
MultiSeg: Semantically Meaningful, Scale-Diverse Segmentations From Minimal User Input (2019, methodology): “One could argue that earlier methods use older backbone architectures, and thus our improvement could be based solely on that (though some use similar advanced architectures: DEXTR [27] utilizes ResNet-101 with a PSP head while ITIS [26] uses an Xception-based DeepLabv3+).”
Interactive Full Image Segmentation by Considering All Regions Jointly (2018, methodology): “Our approach of full image segmentation brings several advantages over modern interactive single object segmentation methods [24, 30, 31, 32, 36, 37, 60]: (I) It enables the annotator to focus on the largest errors in the whole image, rather than on the largest error of one given object.”
Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections (2019, methodology): “We adopt a typical interactive object segmentation process [12,56,38,33,11,29]: the model is given an image and makes an initial foreground / background prediction for every pixel.”
Segment Anything Model for medical image analysis: an experimental study (2023, methodology): “We use a common, intuitive strategy for simulating realistic iterative point prompts, which reflects how those could be generated by a user in an interactive way [24].”
Large-Scale Interactive Object Segmentation With Human Annotators (2019, methodology): “It has been revisited in the recent years under the deep learning umbrella [36, 20, 10, 34, 22, 16, 15, 1], focusing on how to best use clicks and scribbles corrections.”
Content-Aware Multi-Level Guidance for Interactive Instance Segmentation (2019, methodology): “The guidance map helps to select the speciﬁc instance to segment; when working in an iterative setting, it can also help correct errors from previous seg-mentations [6, 32, 35, 53].”