SOLQ: Segmenting Objects by Learning Queries

By Bin Dong, Fangao Zeng, Tiancai Wang, X. Zhang, Yichen Wei. 2021

141 citations recorded by the source corpus.

In this paper, we propose an end-to-end framework for instance segmentation. Based on the recently introduced DETR [1], our method, termed SOLQ, segments objects by learning unified queries. In SOLQ, each query represents one object and has multiple representations: class, location and mask. The object queries learned perform classification, box regression and mask encoding simultaneously in an unified vector form. During training phase, the mask vectors encoded are supervised by the compression coding of raw spatial masks. In inference time, mask vectors produced can be directly transformed to spatial masks by the inverse process of compression coding. Experimental results show that SOLQ can achieve state-of-the-art performance, surpassing most of existing approaches. Moreover, the joint learning of unified query representation can greatly improve the detection performance of DETR. We hope our SOLQ can serve as a strong baseline for the Transformer-based instance segmentation. Code is available at https://github.com/megvii-research/SOLQ.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Generalized Decoding for Pixel, Image, and Language (2022, methodology): “Models for both tasks have evolved from CNN-based architectures [48] to transformer-based ones [11], and from two-stage models [28], one-stage models [3, 66] to the recent query-based approaches [18,92].”
Panoptic SegFormer (2021, unknown): “Recently, SOLQ [31] Backbone .”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, methodology): “Inspired by the advanced instance segmentation methods [5, 32, 34, 6] where an instance mask within the bounding box is represented by a set of learnable parameters, we regard each patch of the BEV map as the mask within a bounding box and parameterize these BEV patches by different object queries.”
Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers (2021, background): “The recent popular endto-end object detection frameworks have inspired many other related works [13, 25].”
A Survey of Visual Transformers (2021, methodology): “Both ISTR and SOLQ obtain comparable results and outperform previous methods even with approximation-based suboptimal embeddings.”
Transformer-Based Visual Segmentation: A Survey (2023, methodology): “Moreover, it also pre-trains a point cloud Tokenizer with a discrete variational autoEncoder to encode the semantic contents and train an extra decoder using the reconstruction loss.”
A Survey on Vision Transformer (2020, methodology): “Some papers such as [243], [52] also discussed using Transformer to deal with segmentation task.”
DETRs with Hybrid Matching (2022, background): “Since the success of pioneering work DEtection TRansformer (DETR) [4] on object detection tasks, DETR-based approaches have achieved significant progress on various fundamental vision recognition tasks such as object detection [46, 53, 77, 83], instance segmentation [11, 12, 22, 73], panoptic segmen”