Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections

By Theodora Kontogianni, Michael Gygli, Jasper R. R. Uijlings, Vittorio Ferrari. 2019

64 citations recorded by the source corpus.

In interactive object segmentation a user collaborates with a computer vision model to segment an object. Recent works employ convolutional neural networks for this task: Given an image and a set of corrections made by the user as input, they output a segmentation mask. These approaches achieve strong performance by training on large datasets but they keep the model parameters unchanged at test time. Instead, we recognize that user corrections can serve as sparse training examples and we propose a method that capitalizes on that idea to update the model parameters on-the-fly to the data at hand. Our approach enables the adaptation to a particular object and its background, to distributions shifts in a test set, to specific object classes, and even to large domain changes, where the imaging modality changes between training and testing. We perform extensive experiments on 8 diverse datasets and show: Compared to a model with frozen parameters, our method reduces the required corrections (i) by 9%-30% when distribution shifts are small between training and testing; (ii) by 12%-44% when specializing to a specific class; (iii) and by 60% and 77% when we completely change domain between training and testing.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “…via various representations: scribbles, clicks, extreme points, etc. Click-based interactive segmentation is one of the most well-studied topics among other deep learning-based interactive segmentation approaches, as it has well-established protocols of training and evaluation [1, 2, 3, 4].”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “[23] employ user corrections as training samples and update the model parameters instantly.”
EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow (2021, methodology): “[24] adjusted the mask of the target object by optimizing the model parameters at test time.”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, methodology): “We compare our results with existing state-of-the-art methods, including fBRS [17], IA+SA [38], FCA [52], and RITM [11].”