Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks

By Abhijit Guha Roy, Nassir Navab, Christian Wachinger. 2018

1,025 citations recorded by the source corpus.

Fully convolutional neural networks (F-CNNs) have set the state-of-the-art in image segmentation for a plethora of applications. Architectural innovations within F-CNNs have mainly focused on improving spatial encoding or network connectivity to aid gradient flow. In this paper, we explore an alternate direction of recalibrating the feature maps adaptively, to boost meaningful features, while suppressing weak ones. We draw inspiration from the recently proposed squeeze & excitation (SE) module for channel recalibration of feature maps for image classification. Towards this end, we introduce three variants of SE modules for image segmentation, (i) squeezing spatially and exciting channel-wise (cSE), (ii) squeezing channel-wise and exciting spatially (sSE) and (iii) concurrent spatial and channel squeeze & excitation (scSE). We effectively incorporate these SE modules within three different state-of-the-art F-CNNs (DenseNet, SD-Net, U-Net) and observe consistent improvement of performance across all architectures, while minimally effecting model complexity. Evaluations are performed on two challenging applications: whole brain segmentation on MRI scans (Multi-Atlas Labelling Challenge Dataset) and organ segmentation on whole body contrast enhanced CT scans (Visceral Dataset).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Rui Huang
5. Le Yang
6. Nassir Navab
7. Hirotoshi Kitamura
8. Yauhen Babakhin

In Computer Vision and Pattern Recognition

1. Yujiu Yang
2. Weihao Xia
3. Shiji Song
4. Ming-Hsuan Yang
5. Gao Huang
6. Luc Van Gool
7. Yujun Shen
8. Dacheng Tao

In Model Architectures

1. Artsiom Sanakoyeu
2. Hirotoshi Kitamura
3. Yauhen Babakhin
4. Nassir Navab
5. Christian Wachinger
6. Abhijit Guha Roy
7. Shiji Song
8. Rui Huang

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, background): “Sa & Sp Image segmentation [34], [129], [146], [148], [150], [154], [156], [216], [217], [218], [219], [220], Image-to-image translation [221], Object detection [110], [111], [147], [148], [164], Semantic image synthesis [222], [223], [224], Computer Image denoising [225], Fine-grained classificatio”
• Multi-Scale Self-Guided Attention for Medical Image Segmentation (2019, background): “Despite the growing interest on integrating attention mechanisms in image segmentation networks for natural scenes, their adoption in medical images remains scarce [23]–[27], being limited to simple attention models.”
• U-Net Transformer: Self and Cross Attention for Medical Image Segmentation (2021, neutral): “Attention mechanisms are a relatively recent problem in medical imaging [16, 8, 10–12].”
• nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation (2018, background): “However, each segmentation benchmark seems to require specialized architectures and training scheme modifications to achieve competitive performance [1,2,3,4,5].”
• High-fidelity GAN Inversion with Padding Space (2022, methodology): “Additionally, we equip the Resblocks with squeeze and excitation block [40] to enhance them.”
• AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images (2022, neutral): “, 2020), scSENet (Roy et al., 2018), SENet (Hu et al.”
• No New-Net (2018, background): “Recently, a growing number of architectural modifications to encoder-decoder networks have been proposed that are designed to improve the performance of the networks for their specific tasks [17,19,20,21,10,6,22,7].”
• Recalibrating Fully Convolutional Networks With Spatial and Channel “Squeeze and Excitation” Blocks (2018, methodology): “This aggregation provides equal importance to the two sSE and cSE methods [13].”
• VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images (2020, methodology): “…maximum intensity projections (MIP) of the localised spine region, and finally, 3) Vertebral segmentation, performed by an improved U-Net (Ronneberger et al., 2015; Roy et al., 2018) to segment vertebral patches, extracted at 1mm resolution, ix around the centroids predicted by the preceding stage.”
• Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks (2019, methodology): “First, we insert concurrent spatial and channel Squeeze & Excitation modules (scSE) [39] after each encoder and decoder block.”
• CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation (2020, background): “[17] proposed concurrent spatial and channel wise ’Squeeze and Excitation’ (scSE) frameworks”