Transformers in Medical Imaging: A Survey

By Fahad Shamshad, Salman Hameed Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, Huan Fu. Medical Image Anal., 2022

1,206 citations recorded by the source corpus.

Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as de facto operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of the applications of Transformers in medical imaging covering various aspects, ranging from recently proposed architectural designs to unsolved issues. Specifically, we survey the use of Transformers in medical image segmentation, detection, classification, restoration, synthesis, registration, clinical report generation, and other tasks. In particular, for each of these applications, we develop taxonomy, identify application-specific challenges as well as provide insights to solve them, and highlight recent trends. Further, we provide a critical discussion of the field's current state as a whole, including the identification of key challenges, open problems, and outlining promising future directions. We hope this survey will ignite further interest in the community and provide researchers with an up-to-date reference regarding applications of Transformer models in medical imaging. Finally, to cope with the rapid development in this field, we intend to regularly update the relevant latest papers and their open-source implementations at https://github.com/fahadshamshad/awesome-transformers-in-medical-imaging.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/24aa57dae649b6683d8f5bc8deaf2ff549cdacc4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

David A. Clifton

In Computer Vision and Pattern Recognition

Citations in the corpus

A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks (2023, background): “(Shamshad et al., 2023) Computer Visionmedical • A review of a number of transformer models with a focus on some tasks related to medical images and different image modalities, and a description of the datasets used for these tasks • This paper compares deep learning models starting with CNNs and mo”
How Artificial Intelligence Is Shaping Medical Imaging Technology: A Survey of Innovations and Applications (2023, background): “These models are often referred to as vision transformers (ViTs) or image transformers [29] and come to introduce performance benefits, especially in noisy conditions [30,31].”
Transformers in Remote Sensing: A Survey (2022, neutral): “[19] survey the use of transformers models”
MedViT: A Robust Vision Transformer for Generalized Medical Image Classification (2023, background): “In addition, long-range dependencies are crucial for medical images because the background in medical images is generally scattered [54], thereby learning longrange dependencies between the pixels corresponding to the background can help the network in preventing misclassification [55].”
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, neutral): “stantial benefits due to its capacity to enrich language understanding and improve model performance across various pretraining tasks [173]–[175].”
MALUNet: A Multi-Attention and Light-weight UNet for Skin Lesion Segmentation (2022, methodology): “Because of its powerful feature extraction ability, researchers recently focused on how to combine ViT and its variants with UNet to obtain better results [27] [28].”
Multimodal Learning With Transformers: A Survey (2022, background): “ing [55], video tasks [56], and vision language pretraining [57].”