k-means Mask Transformer

By Qihang Yu, Huiyu Wang, Siyuan Qiao, Maxwell D. Collins, Yukun Zhu, Hatwig Adam, Alan L. Yuille, Liang-Chieh Chen. 2022

164 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Generalized Decoding for Pixel, Image, and Language (2022, methodology): “In Table 3, we show that we can simply fine-tune the class embedding layer, mask embedding layer or the whole decoder to reach a decent segmentation performance and surpass the fully finetuned tiny SoTA models like kMaX-DeepLab [87].”
CoMFormer: Continual Learning in Semantic and Panoptic Segmentation (2022, methodology): “Recently, methods addressing segmentation as a mask class-siﬁcation problem [11, 12, 50, 56] have been proposed, introducing a transformer architecture able to solve multiple tasks at once.”
Transformer-Based Visual Segmentation: A Survey (2023, methodology): “After that, inspired by the k-means clustering algorithm, kMaX-DeepLab [228] proposes k-means cross-attention by introducing cluster-wise argmax operation in the cross-attention operation.”
DETRs with Hybrid Matching (2022, background): “Inspired by the great success of DETR in object detection, many recent efforts have constructed various DETR-based approaches for segmentation tasks [7, 8, 22, 28, 71, 72] that aim for more accurate pixel localization and recognition, 3D object detection tasks [1, 17, 18, 27, 31, 35, 35, 36, 40, 41,”
Continual Segmentation with Disentangled Objectness Learning and Class Recognition (2024, unknown): “Query-based seg-menters [9, 10, 21, 55] have unified image segmentation via mask classification and solve semantic, instance, and panoptic segmentation using the same framework.”
Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models (2023, result): “However, previous works [5,9–11,39,42,47,48,65,79,86,91] follow a closed closed-vocabulary assumption and only recognize categories present in the training set.”
An Image is Worth 32 Tokens for Reconstruction and Generation (2024, unknown): “…question, we draw inspiration from several image understanding tasks where model predictions are based solely on high-level information extracted from input images —such as in image classification [17], object detection [8, 77], segmentation [64, 71], and multi-modal large language models [1, 40].”