Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation

By Hu Cao, Yueyue Wang, Jieneng Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, Manning Wang. ECCV Workshops, 2021

5,366 citations recorded by the source corpus.

In the past few years, convolutional neural networks (CNNs) have achieved milestones in medical image analysis. Especially, the deep neural networks based on U-shaped architecture and skip-connections have been widely applied in a variety of medical image tasks. However, although CNN has achieved excellent performance, it cannot learn global and long-range semantic information interaction well due to the locality of the convolution operation. In this paper, we propose Swin-Unet, which is an Unet-like pure Transformer for medical image segmentation. The tokenized image patches are fed into the Transformer-based U-shaped Encoder-Decoder architecture with skip-connections for local-global semantic feature learning. Specifically, we use hierarchical Swin Transformer with shifted windows as the encoder to extract context features. And a symmetric Swin Transformer-based decoder with patch expanding layer is designed to perform the up-sampling operation to restore the spatial resolution of the feature maps. Under the direct down-sampling and up-sampling of the inputs and outputs by 4x, experiments on multi-organ and cardiac segmentation tasks demonstrate that the pure Transformer-based U-shaped Encoder-Decoder network outperforms those methods with full-convolution or the combination of transformer and convolution. The codes and trained models will be publicly available at https://github.com/HuCaoFighting/Swin-Unet.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/ea7cfe7f2340584cbe653da6077ee7c213e49b92

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

A Multilevel Multimodal Fusion Transformer for Remote Sensing Semantic Segmentation (2024, methodology): “Empowered with these robust backbones, i.e., CNN and transformers, semantic segmentation for single modality [29], [30], [31], [32] achieved outstanding performance in the CV community.”
Efficient Long-Range Attention Network for Image Super-resolution (2022, methodology): “First, we discard the layer normalization (LN), which is widely employed in previous transformer models [7, 35, 28, 55], because the LN fragments the calculation of SA into many element-wise operations, which are not friendly for efficient inference.”
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction (2021, background): “66,66], segmentation [6,28,55,64,65], human pose estimation [23, 24, 56], etc.”
SwinSUNet: Pure Transformer Network for Remote Sensing Image Change Detection (2022, methodology): “In [42], a new up-sampling method, patch expanding, was proposed.”
VM-UNet: Vision Mamba UNet for Medical Image Segmentation (2024, positive): “The VSS block derived from VMamaba [23] is the core module of VM-UNet, as depicted in Figure 2 (b).”
Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer (2021, methodology): “For a fair comparison, in this work, we employ the open-sourced implementation of UNet (Ron-neberger et al., 2015) and Swin-UNet (Cao et al., 2021) as baselines.”
MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (2022, methodology): “In high-level vision, Transformer has been widely applied in image classification [7, 19, 25, 27, 49, 65], object detection [6, 22, 23, 60, 86, 93], semantic segmentation [16, 69,76,77,90,91], human pose estimation [14,15,32,37,42, 43, 53, 79, 89], etc.”
Diffusion models in medical imaging: A comprehensive survey (2022, background): “images and labels accessible for medical image segmentation is restricted due to the time, cost, and expertise required [107, 108, 109].”
Transformers in Medical Imaging: A Survey (2022, methodology): “Experiments on ACDC and Synapse datasets show that nnFormer surpass Swin-UNet [125] (transformer-based medical segmentation approach) by over 7% (dice score) on Synapse dataset.”
Transformer-Based Visual Segmentation: A Survey (2023, background): “Swin-Unet [346] designs a symmetric Swin-like [23] decoder to recover ﬁne details.”
Comparison of Vision Transformers and Convolutional Neural Networks in Medical Image Analysis: A Systematic Review (2024, neutral): “7) [91].”
Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining (2024, positive): “First, the up-sample block was replaced by 2 × patch expanding layer [3] and two VSS blocks.”
Swin Transformer Embedding UNet for Remote Sensing Image Semantic Segmentation (2022, methodology): “With the Swin transformer as the backbone, Cao et al. [31] and Lin et al. [32] developed the U-shaped encoder–decoder framework for medical image semantic segmentation.”
MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation (2023, background): “Transformers [30,7,21] have seen wide-scale adoption in medical image segmentation as either components of hybrid architectures [3,9,33,2,8,31] or standalone techniques [34,25,15] for state-of-the-art performance.”
Medical Image Segmentation Review: The Success of U-Net (2022, methodology): “(a) Swin Transformer block [48], [156], (b) MISSFormer Enhanced Transformer block [49], (c) D-Former Transformer block [50].”
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis (2021, methodology): “Recent works such as Swin UNet [4] and DS-TransUNet [28] utilize the merits of Swin Transformers for 2D segmentation and achieve promising performance.”