Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images

By Jo Schlemper, Ozan Oktay, Michiel Schaap, Mattias Paul Heinrich, Bernhard Kainz, Ben Glocker, Daniel Rueckert. Medical Image Anal., 2018

1,840 citations recorded by the source corpus.

Graphical abstract Figure. No Caption available. ABSTRACT We propose a novel attention gate (AG) model for medical image analysis that automatically learns to focus on target structures of varying shapes and sizes. Models trained with AGs implicitly learn to suppress irrelevant regions in an input image while highlighting salient features useful for a specific task. This enables us to eliminate the necessity of using explicit external tissue/organ localisation modules when using convolutional neural networks (CNNs). AGs can be easily integrated into standard CNN models such as VGG or U‐Net architectures with minimal computational overhead while increasing the model sensitivity and prediction accuracy. The proposed AG models are evaluated on a variety of tasks, including medical image classification and segmentation. For classification, we demonstrate the use case of AGs in scan plane detection for fetal ultrasound screening. We show that the proposed attention mechanism can provide efficient object localisation while improving the overall prediction performance by reducing false positives. For segmentation, the proposed architecture is evaluated on two large 3D CT abdominal datasets with manual annotations for multiple organs. Experimental results show that AG models consistently improve the prediction performance of the base architectures across different datasets and training sizes while preserving computational efficiency. Moreover, AGs guide the model activations to be focused around salient regions, which provides better insights into how model predictions are made. The source code for the proposed AG models is publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Pediatrics, Perinatology and Child Health

View Paper: https://www.semanticscholar.org/paper/2a2bdf5cf0d73bc333423a8fd246593f4bf65322

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation (2021, methodology): “Some studies have tried to address this problem by using atrous convolutional layers [10,11], self-attention mechanisms [12,13], and image pyramids [14].”
TransBTS: Multimodal Brain Tumor Segmentation Using Transformer (2021, background): “[14] propose an attention gate model, which can be integrated into standard CNN models with minimal computational overhead while increasing the model sensitivity and prediction accuracy.”
Customized Segment Anything Model for Medical Image Segmentation (2023, methodology): “Compared with CNN-based methods [29,17,30], the integration of transformer blocks [4,3,2,1] makes it possible to attend and aggregate similar features in global manner, which leads to unprecedented performance in medical image segmentation.”
Deep semantic segmentation of natural and medical images: a review (2019, methodology): “organ segmentation from MRI images. Similarly, Qin et al. [115] proposed a dilated convolution base block to preserve more detailed attention in 3D medical image segmentation. Similarly, other papers [80,56,77,101,102,105,124] have leveraged the attention concept into medical image segmentation as w”
Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation (2021, methodology): “In contrast, the Attention U-Net (Schlemper et al., 2019) and V-Net (Milletari et al., 2016) leverage the Dice loss, which is based on the most commonly used metric for evaluating segmentation performance, and therefore represents a form of direct loss minimisation.”
Differentiable Top-k with Optimal Transport (2020, background): “This may lead to redundancy of the attention (Zhu et al., 2018; Schlemper et al., 2019).”
Explainable Machine Learning for Scientific Insights and Discoveries (2019, methodology): “Furthermore, [73] builds an attentive gated network that gradually fitted its attention weights with respect to targeted organ boundaries in segmenting tasks.”
Deep learning in medical image registration: a review (2019, methodology): “Recently, attention gate was used in CNN to improve performance in image classification and segmentation [124].”
Reliable Tuberculosis Detection Using Chest X-Ray With Deep Learning, Segmentation and Visualization (2020, background): “Therefore, even though the performance of the CNNs is quite good in classification of TB and normal images, the reliability of these networks would be criticized and several researchers have recently criticized CNNs for learning from non-relevant areas of the image [74].”