TransBTS: Multimodal Brain Tumor Segmentation Using Transformer

By Wenxuan Wang, Chen Chen, Meng Ding, Jiangyun Li, Hong Yu, Sen Zha. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2021

1,166 citations recorded by the source corpus.

Transformer, which can benefit from global (long-range) information modeling using self-attention mechanisms, has been successful in natural language processing and 2D image classification recently. However, both local and global features are crucial for dense prediction tasks, especially for 3D medical image segmentation. In this paper, we for the first time exploit Transformer in 3D CNN for MRI Brain Tumor Segmentation and propose a novel network named TransBTS based on the encoder-decoder structure. To capture the local 3D context information, the encoder first utilizes 3D CNN to extract the volumetric spatial feature maps. Meanwhile, the feature maps are reformed elaborately for tokens that are fed into Transformer for global feature modeling. The decoder leverages the features embedded by Transformer and performs progressive upsampling to predict the detailed segmentation map. Extensive experimental results on both BraTS 2019 and 2020 datasets show that TransBTS achieves comparable or higher results than previous state-of-the-art 3D methods for brain tumor segmentation on 3D MRI scans. The source code is available at https://github.com/Wenxuan-1119/TransBTS

Subfields: Computer Vision and Pattern Recognition, Neurology

View Paper: https://www.semanticscholar.org/paper/147164a3905f41a7a5a10f732d086a621c9c5862

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Neurology

Citations in the corpus

Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation (2021, methodology): “Currently, various combinations of Transformer with CNN are applied in multi-modal brain tumor segmentation [29] and 3D medical image segmentation [1,30].”
MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation (2023, background): “Transformers [30,7,21] have seen wide-scale adoption in medical image segmentation as either components of hybrid architectures [3,9,33,2,8,31] or standalone techniques [34,25,15] for state-of-the-art performance.”
MISSFormer: An Effective Medical Image Segmentation Transformer (2021, methodology): “For example, DeiT(Touvron et al. 2021) explored the efﬁ-cient training strategies for ViT, PVT(Wang et al. 2021c) proposed a pyramid transformer with SRA to reduce the computational complexity, and Swin transformer(Liu et al. 2021) was an efﬁcient and effective hierarchical vision transformer, whose”
MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer (2023, methodology): “the recently popular vision transformers (ViTs) [2,22], to medical image segmentation tasks.”
Transformers in Medical Imaging: A Survey (2022, methodology): “In another work, inspired from the architectural design of TransBTS [138], Jia et al.”
nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation (2024, methodology): “Despite this, numerous studies claim methodological superiority while at least 50% of the benchmark is made up by either BTCV [37,11,9] or BraTS [14,34,31,28,16].”
UNeXt: MLP-based Rapid Medical Image Segmentation Network (2022, methodology): “Other transformer-based networks like MedT [20], TransBTS [25], and UNETR [11] have also been proposed for medical image segmentation.”
Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images (2022, background): “We have compared the performance of Swin UNETR in our internal cross validation split against the winning methologies of previous years such as SegRes-Net [31], nnU-Net [19] and TransBTS [39].”
nnFormer: Interleaved Transformer for Volumetric Segmentation (2021, methodology): “TransBTS [35] first utilized 3D convnets to extract the volumetric spatial features and down-sample the input 3D images to produce hierarchical representations.”
Medical Image Segmentation Review: The Success of U-Net (2022, methodology): “[39] point them out and propose TransBTS as a U-Net-like architecture, modeling local and global information in spatial and slice/depth dimensions.”
Mixed Transformer U-Net for Medical Image Segmentation (2021, background): “Though some of these work achieved satisfying results [7, 9, 10, 11], they usually rely heavily on large-scale pre-training, causing inconvenience to the use of the methods.”
Transformers in Medical Image Analysis: A Review (2022, neutral): “[72], [76], [84], [90], [99], [103], [104], polyp segmentation [79], [87], [101], skin lesion segmentation [79], [87], [91], [101], hip segmentation [79], prostate segmentation [79],”