LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation

By Guoping Xu, Xingrong Wu, Xuan Zhang, Xinwei He. Chinese Conference on Pattern Recognition and Computer Vision, 2021

314 citations recorded by the source corpus.

Medical image segmentation plays an essential role in developing computer-assisted diagnosis and therapy systems, yet still faces many challenges. In the past few years, the popular encoder-decoder architectures based on CNNs (e.g., U-Net) have been successfully applied in the task of medical image segmentation. However, due to the locality of convolution operations, they demonstrate limitations in learning global context and long-range spatial relations. Recently, several researchers try to introduce transformers to both the encoder and decoder components with promising results, but the efficiency requires further improvement due to the high computational complexity of transformers. In this paper, we propose LeViT-UNet, which integrates a LeViT Transformer module into the U-Net architecture, for fast and accurate medical image segmentation. Specifically, we use LeViT as the encoder of the LeViT-UNet, which better trades off the accuracy and efficiency of the Transformer block. Moreover, multi-scale feature maps from transformer blocks and convolutional blocks of LeViT are passed into the decoder via skip-connection, which can effectively reuse the spatial information of the feature maps. Our experiments indicate that the proposed LeViT-UNet achieves better performance comparing to various competing methods on several challenging medical image segmentation benchmarks including Synapse and ACDC. Code and models will be publicly available at https://github.com/apple1986/LeViT_UNet.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/faa30dfcb1531df4e4d5c219bad06d65f6c860fa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

DCSAU-Net: A Deeper and More Compact Split-Attention U-Net for Medical Image Segmentation (2022, methodology): “[31] proposed LeViT-UNet to collect distant spatial information from features.”
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis (2021, background): “Recently, transformer-based networks [24, 47, 49, 55] are proposed for medical image segmentation.”
nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation (2024, background): “Consequently, researchers often resort to custom train/test splits for controlled comparisons against baselines, but these typically involve small test sets that introduce substantial result instability and question the significance of minor performance gains [36,39,41].”
nnFormer: Volumetric Medical Image Segmentation via a 3D Transformer (2021, background): “LeViT-UNet-384s”
Transformers in Medical Image Analysis: A Review (2022, methodology): “Transformer-based methods have also been applied to a variety of segmentation tasks, including abdominal multiorgan segmentation [68], [69], [70], [72], [81], [25], [95], [96],”