GRES: Generalized Referring Expression Segmentation

By Chang Liu, Henghui Ding, Xudong Jiang. Computer Vision and Pattern Recognition, 2023

331 citations recorded by the source corpus.

Referring Expression Segmentation (RES) aims to generate a segmentation mask for the object described by a given language expression. Existing classic RES datasets and methods commonly support single-target expressions only, i.e., one expression refers to one target object. Multitarget and no-target expressions are not considered. This limits the usage of RES in practice. In this paper, we introduce a new benchmark called Generalized Referring Expression Segmentation (GRES), which extends the classic RES to allow expressions to refer to an arbitrary number of target objects. Towards this, we construct the first largescale GRES dataset called gRefCOCO that contains multitarget, no-target, and single-target expressions. GRES and gRefCOCO are designed to be well-compatible with RES, facilitating extensive experiments to study the performance gap of the existing RES methods on the GRES task. In the experimental study, we find that one of the big challenges of GRES is complex relationship modeling. Based on this, we propose a region-based GRES baseline ReLA that adaptively divides the image into regions with subinstance clues, and explicitly models the region-region and region-language dependencies. The proposed approach ReLA achieves new state-of-the-art performance on the both newly proposed GRES and classic RES tasks. The proposed gRefCOCO dataset and method are available at https://henghuiding.github.io/GRES.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1ec4bc98fafa8d338f676f7a1b1b1131e8ca978e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Chen Change Loy
  2. Ziwei Liu
  3. Kai Chen
  4. Jiangmiao Pang
  5. Wenwei Zhang
  6. Henghui Ding
  7. Haobo Yuan
  8. Guangliang Cheng

In Computer Vision and Pattern Recognition

  1. Chen Change Loy
  2. Kai Chen
  3. Ziwei Liu
  4. Luc Van Gool
  5. Jiangmiao Pang
  6. Xiaogang Wang
  7. Henghui Ding
  8. Dacheng Tao

Citations in the corpus