Referring Image Segmentation via Recurrent Refinement Networks

By Ruiyu Li, Kaican Li, Yi-Chun Kuo, Michelle Shu, Xiaojuan Qi, Xiaoyong Shen, Jiaya Jia. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

294 citations recorded by the source corpus.

We address the problem of image segmentation from natural language descriptions. Existing deep learning-based methods encode image representations based on the output of the last convolutional layer. One general issue is that the resulting image representation lacks multi-scale semantics, which are key components in advanced segmentation systems. In this paper, we utilize the feature pyramids inherently existing in convolutional neural networks to capture the semantics at different scales. To produce suitable information flow through the path of feature hierarchy, we propose Recurrent Refinement Network (RRN) that takes pyramidal features as input to refine the segmentation mask progressively. Experimental results on four available datasets show that our approach outperforms multiple baselines and state-of-the-art1.

View Paper: https://www.semanticscholar.org/paper/8f44799620bf22f6efa01c35497de7dda3e5d4ab

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click (2020, methodology): “Methods RMI [35] DMN [42] RNN [30] MAttNet [61] CMSA [60] Phrase PhraseClick”
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language (2019, methodology): “In contrast, other methods [19, 39, 63] address the referring expression task with a single stage proposing end-to-end approaches, and some [19, 32, 30, 36, 64, 4] produce a segmentation mask for the object description.”
Vision-Language Transformer and Query Generation for Referring Segmentation (2021, background): “Referring segmentation targets to generate segmentation mask for the target object referred by a given query expression in natural language [10, 16, 15, 3].”
An Underwater Image Enhancement Benchmark Dataset and Beyond (2019, background): “Additionally, lacking sufficient and effective training data, the performance of deep learning-based underwater image enhancement algorithms does not match the success of recent deep learning-based high-level [10], [11] and low-level [12], [13] vision problems.”
Image Segmentation Using Text and Image Prompts (2021, background): “Early approaches used recurrent networks in combination with CNNs to address this problem [15, 16, 17, 18].”
MDETR - Modulated Detection for End-to-End Multi-Modal Understanding (2021, background): “A separate line of work that attacks a similar problem to ours but with a much more task specialized model architectures are the single [59, 4, 23] and two stage [39, 12] referring expression segmentation and phrase detection models which are designed specifically for this task.”
Multi-Task Collaborative Network for Joint Referring Expression Comprehension and Segmentation (2020, methodology): “, RefCOCO [13], RefCOCO+ [13] and RefCOCOg [24], and compare them to a set of state-of-theart methods [43, 37, 38, 40, 16] of both REC and RES.”
URVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark (2020, methodology): “A na¨ıve baseline for the task is applying referring image segmentation techniques [16, 12, 37, 36] to each input frame independently.”