Interactive Image Segmentation With First Click Attention

By Zheng Lin, Zhao Zhang, Lin-Zhuo Chen, Ming-Ming Cheng, Shao-Ping Lu. 2020

165 citations recorded by the source corpus.

In the task of interactive image segmentation, users initially click one point to segment the main body of the target object and then provide more points on mislabeled regions iteratively for a precise segmentation. Existing methods treat all interaction points indiscriminately, ignoring the difference between the first click and the remaining ones. In this paper, we demonstrate the critical role of the first click about providing the location and main body information of the target object. A deep framework, named First Click Attention Network (FCA-Net), is proposed to make better use of the first click. In this network, the interactive segmentation result can be much improved with the following benefits: focus invariance, location guidance, and error-tolerant ability. We then put forward a click-based loss function and a structural integrity strategy for better segmentation effect. The visualized segmentation results and sufficient experiments on five datasets demonstrate the importance of the first click and the superiority of our FCA-Net.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, background): “88 ♪ FCA-Net [28] CVPR20 ResNet-101 - 2.”
FocalClick: Towards Practical Interactive Image Segmentation (2022, background): “The FLOPs of our B0-S1 version is 15 times smaller than the lightest RITM [36], 360 times smaller than FCANet [27].”
Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “Most recent works on click-based interactive segmentation [2, 8, 3] simulate user clicks during training via the random sampling strategy that ignores any relations between the clicks.”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, result): “We compare our results with existing state-of-the-art methods, including fBRS [17], IA+SA [38], FCA [52], and RITM [11].”
Interactive Object Segmentation With Inside-Outside Guidance (2022, background): “Existing interactive object segmentation algorithms [1], [49], [91] are typically trained on the combination of a finely annotated PASCAL VOC [3] and a coarsely annotated SBD [89] dataset (Fig.”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “For annotation simulation in the global view, we follow the strategy in [31].”
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic (2023, methodology): “There are also some methods (Lin et al., 2020, 2022) that use 0/1 mask or Gaussian map input with the original image to emphasize the area of user interest.”
Conditional Diffusion for Interactive Segmentation (2021, methodology): “PDM refines the boundary with low-level consistency, which helps to capture fine details as shown in zoomed-in patches.”
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders (2020, methodology): “Furthermore, we plan to capture new datasets with multiple human annotations to further model the statistics of human uncertainty in interactive image segmentation [37], camouflaged object detection [16], etc.”
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation (2023, unknown): “We refer to (Lin et al. 2020) for random sampling generation and (Mahade-van, Voigtlaender, and Leibe 2018) for simulating the iterative sampling process.”