Interactive Object Segmentation With Inside-Outside Guidance

By Shiyin Zhang, J. Liew, Yunchao Wei, Shikui Wei, Yao Zhao. 2020

135 citations recorded by the source corpus.

This paper explores how to harvest precise object segmentation masks while minimizing the human interaction cost. To achieve this, we propose an Inside-Outside Guidance (IOG) approach in this work. Concretely, we leverage an inside point that is clicked near the object center and two outside points at the symmetrical corner locations (top-left and bottom-right or top-right and bottom-left) of a tight bounding box that encloses the target object. This results in a total of one foreground click and four background clicks for segmentation. The advantages of our IOG is four-fold: 1) the two outside points can help to remove distractions from other objects or background; 2) the inside point can help to eliminate the unrelated regions inside the bounding box; 3) the inside and outside points are easily identified, reducing the confusion raised by the state-of-the-art DEXTR in labeling some extreme samples; 4) our approach naturally supports additional clicks annotations for further correction. Despite its simplicity, our IOG not only achieves state-of-the-art performance on several popular benchmarks, but also demonstrates strong generalization capability across different domains such as street scenes, aerial imagery and medical images, without fine-tuning. In addition, we also propose a simple two-stage solution that enables our IOG to produce high quality instance segmentation masks from existing datasets with off-the-shelf bounding boxes such as ImageNet and Open Images, demonstrating the superiority of our IOG as an annotation tool.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Labkit: Labeling and Segmentation Toolkit for Big Image Data (2021, background): “…deep learning with interactive user guidance, for instance clicks on the extreme points of objects (Maninis et al., 2018), inside-outside guidance (Zhang et al., 2020), clicks within objects and boundaries that are iteratively reﬁned (Luo et al., 2021) and a combination of clicks and squiggles…”
SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, background): “Much research has been devoted to explore interactive image segmentation with diﬀerent interaction types, such as bounding boxes [47], polygons [1], clicks [43], scribbles [45], and their combinations [51].”
Interactive Object Segmentation With Inside-Outside Guidance (2022, background): “This paper is an extension of our previous conference version [42].”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, methodology): “Inside outside guidance (IOG) [13] addresses this issue by combining clicks with bounding points of the target object and by allowing corrections of the predicted mask.”
Few-Shot Segmentation via Cycle-Consistent Transformer (2021, background): “However, labeling dense segmentation masks are very time-consuming [45].”
EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow (2021, background): “Therefore, interactive segmentation appears to be an attractive and efficient way, which allows the human annotators to quickly extract the object-of-interest [52].”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, background): “The combination of interactions, such as the bounding box and clicks [4, 52], is also popular in the field.”