F-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation

By Konstantin Sofiiuk, Ilya A. Petrov, Olga Barinova, Anton Konushin. 2020

237 citations recorded by the source corpus.

Deep neural networks have become a mainstream approach to interactive segmentation. As we show in our experiments, while for some images a trained network provides accurate segmentation result with just a few clicks, for some unknown objects it cannot achieve satisfactory result even with a large amount of user input. Recently proposed backpropagating refinement scheme (BRS) introduces an optimization problem for interactive segmentation that results in significantly better performance for the hard cases. At the same time, BRS requires running forward and backward pass through a deep network several times that leads to significantly increased computational budget per click compared to other methods. We propose f-BRS (feature backpropagating refinement scheme) that solves an optimization problem with respect to auxiliary variables instead of the network inputs, and requires running forward and backward passes just for a small part of a network. Experiments on GrabCut, Berkeley, DAVIS and SBD datasets set new state-of-the-art at an order of magnitude lower time per click compared to original BRS. The code and trained models are available at https://github.com/saic-vul/fbrs_interactive_segmentation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “Most recent works on click-based interactive segmentation [2, 8, 3] simulate user clicks during training via the random sampling strategy that ignores any relations between the clicks.”
Conditional Diffusion for Interactive Segmentation (2021, methodology): “BRS [11] uses backward propagation to finetune the guidance map in an online manner. f-BRS [27] refines the intermediate feature to get more precise masks with faster speed compared with BRS.”
Deep Interactive Thin Object Selection (2021, methodology): “On this task, deep learning based methods [51, 23, 50, 32, 21, 15, 31, 16, 27, 41] have been very successful.”
Interactive Object Segmentation With Inside-Outside Guidance (2022, methodology): “According to the human inputs, the interactive segmentation approached can be roughly divided into click-based [1], [37], [38], [47], [48], [49], [50], [51] and polygon-based [52], [53], [54].”
Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections (2019, methodology): “F-BRS [52] (Concurrent Work) 2.”
EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow (2021, methodology): “NoC indicates the number of interactive clicks for achieving over the specified IoU threshold, which is commonly set as 85% and 90% [47, 37, 46, 21], i.”
BEHAVE: Dataset and Method for Tracking Human Object Interactions (2022, methodology): “Human segmentation and SMPL ﬁtting We segment the human in our images by running DetectronV2 [79] followed by manual correction with [67] on the segmentation masks.”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “For the latter, researchers have explored the interaction ambiguity [9,26,30], input information [31,35], backprop-agating [20,41], etc .”