Conditional Diffusion for Interactive Segmentation

By Xi Chen, Zhiyan Zhao, Feiwu Yu, Yilei Zhang, Manni Duan. 2021

103 citations recorded by the source corpus.

In click-based interactive segmentation, the mask extraction process is dictated by positive/negative user clicks; however, most existing methods do not fully exploit the user cues, requiring excessive numbers of clicks for satisfactory results. We propose Conditional Diffusion Network (CDNet), which propagates labeled representations from clicks to conditioned destinations with two levels of affinities: Feature Diffusion Module (FDM) spreads features from clicks to potential target regions with global similarity; Pixel Diffusion Module (PDM) diffuses the predicted logits of clicks within locally connected regions. Thus, the information inferred by user clicks could be generalized to proper destinations. In addition, we put forward Diversified Training (DT), which reduces the optimization ambiguity caused by click simulation. With FDM,PDM and DT, CDNet could better understand user’s intentions and make better predictions with limited interactions. CDNet achieves state-of-the-art performance on several benchmarks.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, methodology): “Similar training pipelines have been proposed by RITM [40] and its follow-up works [8, 9, 31].”
FocalClick: Towards Practical Interactive Image Segmentation (2022, background): “CDNet [5] introduces self-attention into interactive segmentation to predict more consistent results.”
AnyDoor: Zero-shot Object-level Image Customization (2023, unknown): “The segmentor model could be either automatic [27, 39] or interactive [11, 12, 32].”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, methodology): “[9] introduce a nonlocal method to fully exploit the user cues.”
Personalize Segment Anything Model with One Shot (2023, background): “…(He et al., 2017; Wang et al., 2020; Tian et al., 2020); panoptic segmentation, assigning both class labels and instance identification (Kirillov et al., 2019; Li et al., 2019); and interactive segmentation, involving human intervention for refinement (Hao et al., 2021; Chen et al., 2021).”