A Fully Convolutional Two-Stream Fusion Network for Interactive Image Segmentation

By Yang Hu, Andrea Soltoggio, Russell Lock, Steve Carter. 2018

104 citations recorded by the source corpus.

In this paper, we propose a novel fully convolutional two-stream fusion network (FCTSFN) for interactiveimage segmentation. The proposed network includes two sub-networks: a two-stream late fusion network (TSLFN) that predicts the foreground at a reduced resolution, and a multi-scale refining network (MSRN) that refines the foreground at full resolution. The TSLFN includes two distinct deep streams followed by a fusion network. The intuition is that, since user interactions are more direct information on foreground/background than the image itself, the two-stream structure of the TSLFN reduces the number of layers between the pure user interaction features and the network output, allowing the user interactions to have a more direct impact on the segmentation result. The MSRN fuses the features from different layers of TSLFN with different scales, in order to seek the local to global information on the foreground to refine the segmentation result at full resolution. We conduct comprehensive experiments on four benchmark datasets. The results show that the proposed network achieves competitive performance compared to current state-of-the-art interactive image segmentation methods. 1.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Getting to 99% Accuracy in Interactive Segmentation (2020, methodology): “In most prior works (including iFCN [6], RISNET [13], DEXTR [8], IIS-LD [11], FCTSFN [10]), when a user adds a new click, the previous prediction is discarded and a new prediction is make afresh.”
Deep Interactive Thin Object Selection (2021, methodology): “On this task, deep learning based methods [51, 23, 50, 32, 21, 15, 31, 16, 27, 41] have been very successful.”
Interactive Image Segmentation With First Click Attention (2020, methodology): “Following [24, 25, 30, 31, 36, 37, 47], we employ the Mean Intersection Over Union (mIoU) as a metric.”
Large-Scale Interactive Object Segmentation With Human Annotators (2019, background): “It has been revisited in the recent years under the deep learning umbrella [36, 20, 10, 34, 22, 16, 15, 1], focusing on how to best use clicks and scribbles corrections.”
Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections (2019, methodology): “ssical methods are based on a weak appearance model which is specialized to one speciﬁc image. Recent methods rely on Convolutional Neural Networks (CNNs) to interactively produce a segmentation mask [52, 33,10,35,31,15,27,28,3]. These methods take the image and user corrections (transformed into a ”
Content-Aware Multi-Level Guidance for Interactive Instance Segmentation (2019, background): “Existing interactive instance segmentation [26, 31, 32, 53] techniques do not utilize any image information when generating guidance maps (second column).”
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click (2020, background): “Hu et al. [25] employ a two-stream network to deal with image and user clicks separately to make the user interaction impact the result more directly.”
Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey (2020, background): “Recently, in [86], the authors have used fully convolutional two stream fusion network for interactive image segmentation.”
FocusCut: Diving into a Focus View in Interactive Segmentation (2022, background): “[17] provide a two-branch architecture for this task.”
Interactive Full Image Segmentation by Considering All Regions Jointly (2018, methodology): “Our approach of full image segmentation brings several advantages over modern interactive single object segmentation methods [24, 30, 31, 32, 36, 37, 60]: (I) It enables the annotator to focus on the largest errors in the whole image, rather than on the largest error of one given object.”
Interactive Object Segmentation With Inside-Outside Guidance (2020, unknown): “7 FCTSFN [28] 4.”
Understanding Deep Learning Techniques for Image Segmentation (2019, background): “One of the most straight forward implementations of interactive segmentation is to have two parallel branches, one from the image another from the an image representing the interactive stream, and fuse them to perform the segmentation[83].”
MultiSeg: Semantically Meaningful, Scale-Diverse Segmentations From Minimal User Input (2019, methodology): “Existing deep learning-based interactive image segmentation methods [16, 18, 23, 26, 27, 36] have demonstrated significant improvement over the previous techniques using”