Reviving Iterative Training with Mask Guidance for Interactive Segmentation

By Konstantin Sofiiuk, Ilya A. Petrov, Anton Konushin. 2021

294 citations recorded by the source corpus.

Recent works on click-based interactive segmentation have demonstrated state-of-the-art results by using various inference-time optimization schemes. These methods are significantly more computationally expensive than feedforward approaches, as they run backward gradient passes during inference. Moreover, backward passes are not supported in popular mobile frameworks, which complicates the deployment of such methods on embedded devices. In this paper, we study design choices for interactive segmentation and discover that state-of-the-art results can be obtained without any additional optimization schemes. We propose a simple feedforward model for click-based interactive segmentation that employs the segmentation masks from previous steps. It allows not only segmenting an entirely new object but also correcting an existing mask. We analyze the performance of models trained on different datasets and observe that the choice of a training dataset has a large impact on the quality of interactive segmentation. We find that the models trained on a combination of COCO and LVIS with diverse and high-quality annotations outperform all existing models. The code and trained models are available at https://github.com/saic-vul/ritm_interactive_segmentation.

Subfields: Computer Vision and Pattern Recognition, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Optimization & Training

Citations in the corpus

SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, methodology): “Much research has been devoted to exploring interactive image segmentation with different interaction types, such as bounding boxes [45], polygons [1], clicks [40], scribbles [43], and their combinations [49].”
FocalClick: Towards Practical Interactive Image Segmentation (2022, methodology): “For the strategy of click simulation, we exploit iterative training [29] following RITM [36].”
SAM 2: Segment Anything in Images and Videos (2024, methodology): “Similar to previous work (Kirillov et al., 2023; Sofiiuk et al., 2022), we simulate interactive prompting of the model.”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, methodology): “While existing methods in this area have shown improved performance, e.g ., using inference-time optimization schemes [17] or iterative mask-guided training schemes [11], these methods still require users to provide all the clicks.”
Segment Anything Model for medical image analysis: an experimental study (2023, methodology): “We show the average performances of SAM, RITM, SimpleClick, and FocalClick in Figure 5.”