SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation

By Meng-Hao Guo, Chenggang Lu, Qibin Hou, Zheng-Ning Liu, Ming-Ming Cheng, Shiyong Hu. Neural Information Processing Systems, 2022

1,260 citations recorded by the source corpus.

We present SegNeXt, a simple convolutional network architecture for semantic segmentation. Recent transformer-based models have dominated the field of semantic segmentation due to the efficiency of self-attention in encoding spatial information. In this paper, we show that convolutional attention is a more efficient and effective way to encode contextual information than the self-attention mechanism in transformers. By re-examining the characteristics owned by successful segmentation models, we discover several key components leading to the performance improvement of segmentation models. This motivates us to design a novel convolutional attention network that uses cheap convolutional operations. Without bells and whistles, our SegNeXt significantly improves the performance of previous state-of-the-art methods on popular benchmarks, including ADE20K, Cityscapes, COCO-Stuff, Pascal VOC, Pascal Context, and iSAID. Notably, SegNeXt outperforms EfficientNet-L2 w/ NAS-FPN and achieves 90.6% mIoU on the Pascal VOC 2012 test leaderboard using only 1/10 parameters of it. On average, SegNeXt achieves about 2.0% mIoU improvements compared to the state-of-the-art methods on the ADE20K datasets with the same or fewer computations. Code is available at https://github.com/uyzhang/JSeg (Jittor) and https://github.com/Visual-Attention-Network/SegNeXt (Pytorch).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3b2c63b63478d192543bdfb7df7d9a4e1b8d3950

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ziwei Liu
2. Song Han
3. Chuang Gan
4. Chen Change Loy
5. Han Cai
6. Shuicheng Yan
7. Xinchao Wang
8. Kai Chen

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Chen Change Loy
4. Shuicheng Yan
5. Xiaogang Wang
6. Jiashi Feng
7. Ming-Hsuan Yang
8. Yi Yang

Citations in the corpus

• Poly Kernel Inception Network for Remote Sensing Detection (2024, background): “Some [8, 17, 79] adopt multi-scale convolution kernels in order to address challenges across various fields, yet research in remote sensing detection remains scarce.”
• EfficientViT: Lightweight Multi-Scale Attention for High-Resolution Dense Prediction (2023, background): “SegFormer [51] HRFormer [55] SegNeXt [18] EfficientViT We summarize the comparison between our work and prior SOTA high-resolution dense prediction models in Table 1.”
• InceptionNeXt: When Inception Meets ConvNeXt (2023, methodology): “…convolution (DW-Conv) and depth-wise dilation convolution to obtain 21 × 21 receptive filed; FocalNets [76] employ a gating mechanism to fuse multi-level features from stacking depthwise convolutions; Seg-NeXt [17] learns multi-scale features by multiple branches of staking 1 × k and k × 1 .”
• Transformer-Based Visual Segmentation: A Survey (2023, methodology): “Motivated by ConvNeXt, SegNext [145] designs a CNN-like backbone with linear self-attention and performs strongly on multiple SS benchmarks.”