The Importance of Skip Connections in Biomedical Image Segmentation

By Michal Drozdzal, Eugene Vorontsov, Gabriel Chartrand, Samuel Kadoury, C. Pal. 2016

1,139 citations recorded by the source corpus.

In this paper, we study the influence of both long and short skip connections on Fully Convolutional Networks (FCN) for biomedical image segmentation. In standard FCNs, only long skip connections are used to skip features from the contracting path to the expanding path in order to recover spatial information lost during downsampling. We extend FCNs by adding short skip connections, that are similar to the ones introduced in residual networks, in order to build very deep FCNs (of hundreds of layers). A review of the gradient flow confirms that for a very deep FCN it is beneficial to have both long and short skip connections. Finally, we show that a very deep FCN can achieve near-to-state-of-the-art results on the EM dataset without any further post-processing.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “Therefore, we can borrow from the semantic segmentation literature and train the feed-forward set predictor with a soft structured prediction loss, such as the soft intersectionover-union (sIoU) [10], in order to take into account dependencies among elements in the set.”
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation (2016, methodology): “More recently, [8] presented a thorough analysis on the combination of identity mapping [11] and long skip connections [27] for semantic segmentation.”
Resource Optimized Neural Architecture Search for 3D Medical Image Segmentation (2019, methodology): “Most researchers manually determine the best 2D or 3D input patch size, type of down sampling operations along with their stride, and skip connection methods, taking into account available GPU memory and receptive field size [3,4,5].”
NAS-Unet: Neural Architecture Search for Medical Image Segmentation (2019, background): “[31] make a different between long skip connections (which means a skip connection between two feature maps that are far away) and short skip connections (which usually called single residual block) and find that both of them to be beneficial in creating deep architectures for medical image segmenta”
A survey on deep learning in medical image analysis (2017, background): “Drozdzal et al. (2016) Segmentation of neuronal structures EM fCNN with skip connections”
Learning Steerable Filters for Rotation Equivariant CNNs (2017, neutral): “[28] 0.”
Rotation Equivariant Vector Field Networks (2016, neutral): “Our rotation equivariant prediction provides results comparable to other stateof-the-art methods only relying on the raw CNN softmax output [8, 9, 22] (see Table 6).”
Recurrent residual U-Net for medical image segmentation (2019, background): “34 The deep contour-aware network had been proposed in 2016, which can extract multilevel contextual features using a hierarchical architecture for accurate gland segmentation of histology images, and it shows very good performance for segmentation.”
Automatic Brain Tumor Detection and Segmentation Using U-Net Based Fully Convolutional Networks (2017, background): “and is also applicable to biomedical images [28] .”
UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer (2021, methodology): “…mechanism was ﬁrst proposed in U-Net (Ronneberger, Fischer, and Brox 2015), which was designed to bridge the semantic gap between encoder and de-coder, and have proven to be effective in recovering ﬁne-grained details of the target objects (Drozdzal et al. 2016; He et al. 2016; Huang et al. 2017).”
Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images (2018, neutral): “For the segmentation models, the class imbalance problem is tackled using the Sorensen-Dice loss (Drozdzal et al., 2016; Milletari et al., 2016) defined over all semantic classes.”
On the Iterative Refinement of Densely Connected Representation Levels for Semantic Segmentation (2018, methodology): “Top performing classification networks have been successfully extended to perform semantic segmentation [43, 34, 9, 44, 21].”
Sharpness-Aware Low-Dose CT Denoising Using Conditional Generative Adversarial Network (2017, background): “the gradient flow from deep layer to the shallow layer [20].”
Automated Training of Deep Convolutional Neural Networks for Cell Segmentation (2017, background): “13 showed that both long and short skip connections gave better performance in biological applications.”
INet: Convolutional Networks for Biomedical Image Segmentation (2021, background): “This concatenation aims to provide high-level spatial local information together with high-level semantic global information, which has been shown to be important for biomedical image segmentation [21].”