Multi-Scale Self-Guided Attention for Medical Image Segmentation

By Ashish Sinha, Jose Dolz. IEEE journal of biomedical and health informatics, 2019

509 citations recorded by the source corpus.

Even though convolutional neural networks (CNNs) are driving progress in medical image segmentation, standard models still have some drawbacks. First, the use of multi-scale approaches, i.e., encoder-decoder architectures, leads to a redundant use of information, where similar low-level features are extracted multiple times at multiple scales. Second, long-range feature dependencies are not efficiently modeled, resulting in non-optimal discriminative feature representations associated with each semantic class. In this paper we attempt to overcome these limitations with the proposed architecture, by capturing richer contextual dependencies based on the use of guided self-attention mechanisms. This approach is able to integrate local features with their corresponding global dependencies, as well as highlight interdependent channel maps in an adaptive manner. Further, the additional loss between different modules guides the attention mechanisms to neglect irrelevant information and focus on more discriminant regions of the image by emphasizing relevant feature associations. We evaluate the proposed model in the context of semantic segmentation on three different datasets: abdominal organs, cardiovascular structures and brain tumors. A series of ablation experiments support the importance of these attention modules in the proposed architecture. In addition, compared to other state-of-the-art segmentation networks our model yields better segmentation performance, increasing the accuracy of the predictions while reducing the standard deviation. This demonstrates the efficiency of our approach to generate precise and reliable automatic segmentations of medical images. Our code is made publicly available at: https://github.com/sinAshish/Multi-Scale-Attention.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/00a43fd83bef341da691695dc9acd3c9f6c4338f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Cross-level Feature Aggregation Network for Polyp Segmentation (2023, background): “For example, Sinha et al. [38] adopted a multi-scale strategy to incorporate semantic information at different levels for aggregating the relevant contextual features.”
Channel prior convolutional attention for medical image segmentation (2023, unknown): “Richer context dependencies are captured through the utilization of guided self-attention mechanisms, as proposed by Sinha and Dolz (2021).”
Medical Transformer: Gated Axial-Attention for Medical Image Segmentation (2021, background): “The closest works are the ones that use attention mechanisms to boost the performance [29,20,39].”
A General Survey on Attention Mechanisms in Deep Learning (2022, methodology): “In [76], self-attention is used to design a state-of-the-art medical”
How Artificial Intelligence Is Shaping Medical Imaging Technology: A Survey of Innovations and Applications (2023, methodology): “This allows the architecture to model relationships and dependencies between elements within the same sequence, making it particularly useful for tasks that involve capturing long-range dependencies and context [7,40,41].”
Hierarchical Multi-Scale Attention for Semantic Segmentation (2020, methodology): “Furthermore, our method is orthogonal to other attention or pyramid pooling methods such as [22, 25, 26, 9, 27, 10, 28] as these methods use single scale image and perform attention to better combine multi-level features for generating high-resolution predictions.”
UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation (2021, background): “Preliminary studies with simple forms of self-attention have shown its usefulness in segmentation [4,16], detection [24] and reconstruction [9].”
Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, unknown): “approaches [26].”
MISSFormer: An Effective Medical Image Segmentation Transformer (2021, methodology): “And some recent works(Xie et al. 2021b; Mou et al. 2019; Chen et al. 2021; Sinha and Dolz 2020) tried to employ few self-attention layers or transformer layers(Vaswani et al. 2017) in high-level semantic feature maps due to the quadratic relationship between self-attention computational complexity…”
MISSFormer: An Effective Transformer for 2D Medical Image Segmentation (2022, methodology): “…of method uses dilated convolution [9], [10] and pyramid pooling [11] to enlarge the receptive ﬁeld as much as possible, and another type [12], [13] tries to employ few self-attention layers in high-level semantic feature maps, due to the quadratic relationship between self-attention…”
U-Net Transformer: Self and Cross Attention for Medical Image Segmentation (2021, unknown): “Recent works successfully tackle this aspect through Dual attention networks [12, 5] proving the importance of full range attention but to the cost of large parameter overhead and multiple concurrent loss functions.”
Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining (2024, unknown): “However, accurate medical image segmentation requires integrating local features with their corresponding global dependencies [24].”
CHAOS Challenge - Combined (CT-MR) Healthy Abdominal Organ Segmentation (2020, methodology): “rted results for this task. Considering an attention-based model (Wang et al., 2018) as the baseline (DICE: 0.83 0.06), an ablation study is carried out to that increase the segmentation performance (Sinha and Dolz, 2020). The limitations of the baseline is tried to be improved by capturing richer c”
Attentional Feature Fusion (2020, background): “In the ﬁrst type, the features at multiple scales or their concatenated result are fed into the attention module to generate multi-scale attention maps, while the scale of feature context aggregation inside the attention module remains single [2, 3, 46, 6, 36, 41].”