UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation

By Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xianhua Han, Yenwei Chen, Jian Wu. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2020

2,706 citations recorded by the source corpus.

Recently, a growing interest has been seen in deep learning-based semantic segmentation. UNet, which is one of deep learning networks with an encoder-decoder architecture, is widely used in medical image segmentation. Combining multi-scale features is one of important factors for accurate segmentation. UNet++ was developed as a modified Unet by designing an architecture with nested and dense skip connections. However, it does not explore sufficient information from full scales and there is still a large room for improvement. In this paper, we propose a novel UNet 3+, which takes advantage of full-scale skip connections and deep supervisions. The full-scale skip connections incorporate low-level details with high-level semantics from feature maps in different scales; while the deep supervision learns hierarchical representations from the full-scale aggregated feature maps. The proposed method is especially benefiting for organs that appear at varying scales. In addition to accuracy improvements, the proposed UNet 3+ can reduce the network parameters to improve the computation efficiency. We further propose a hybrid loss function and devise a classification-guided module to enhance the organ boundary and reduce the over-segmentation in a non-organ image, yielding more accurate segmentation results. The effectiveness of the proposed method is demonstrated on two datasets. The code is available at: github.com/ZJUGiveLab/UNet-Version

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/0b444f74dd9cc06c2833dd15f9258ef5e169e6ea

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

UNETR++: Delving Into Efficient and Accurate 3D Medical Image Segmentation (2022, methodology): “CNN-based Segmentation Methods : Since the introduction of the U-Net design [28], several CNN-based approaches [2, 14, 37, 38] have extended the standard U-Net architecture for various medical image segmentation tasks.”
AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images (2022, background): “The powerful nonlinear learning ability makes full convolution network (FCN) and U-net have achieve great success in medical images segmentation (Badrinarayanan et al., 2017; Chen et al., 2022b,c; Huang et al., 2020; Long et al., 2015; Ronneberger et al., 2015; Xian et al., 2015; Zhou et al., 2020; ”
Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, methodology): “In the recent literature [63-65], the plain skip connections in U-Net are substituted by more subtle and elaborate skip connections which reduce the semantic gap between the encoder and decoder.”
Medical Image Segmentation Review: The Success of U-Net (2022, background): “SCE Attention U-Net [70] UNet++ [11], [67] RA-Unet [71] BCDU-Net [74] U-Net3+ [68] BiO-Net [69] Skip connections are defined as connections in a neural network that do not connect two following layers but instead skip over at least one layer.”
MISSFormer: An Effective Medical Image Segmentation Transformer (2021, background): “According to this elegant architecture, the variants of U-Net(Isensee et al. 2021; Zhou et al. 2018; Huang et al. 2020) have been achieved excellent performance and impressive results.”
U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation (2024, methodology): “Moreover, techniques such as attention mechanism [76] and multi-scale feature fusion [31] are widely used in medical image segmentation tasks.”
Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation (2021, methodology): “Following this technical route, many algorithms such as 3D U-Net [6], Res-UNet [7], U-Net++ [8] and UNet3+ [9] have been developed for image and volumetric segmentation of various medical imaging modalities.”
DS-TransUNet: Dual Swin Transformer U-Net for Medical Image Segmentation (2021, methodology): “Following the popularity of U-Net, many novel models have been proposed such as UNet++ [2], Res-UNet [3], Attention U-Net [4], DenseUNet [5], R2U-Net [6], KiU-Net [7] and UNet 3+ [8], which are specially designed for medical image segmentation and achieve expressive performance.”
DCSAU-Net: A Deeper and More Compact Split-Attention U-Net for Medical Image Segmentation (2022, methodology): “…0.945±0.078 0 .834±0.189 0.912±0.163 0.848±0.160 0 .762±0.189 DoubleU-Net [22] 0.953±0.092 0 .903±0.149 0.897±0.186 0.879±0.167 0.813±0.191 UNet3+ [21] 0.956±0.068 0 .889±0.151 0.916±0.130 0 .886±0.132 0.816±0.165 TransUNet [32] 0.945±0.085 0 .847±0.186 0.898±0.185 0.849±0.178 0.770±0.203 LeViT-U…”
Medical Image Segmentation via Cascaded Attention Decoding (2023, methodology): “[15] propose UNet 3+ utilizing full-scale skip connections including intra-connections among the decoder blocks.”
EMCAD: Efficient Multi-Scale Convolutional Attention Decoding for Medical Image Segmentation (2024, background): “A variety of U-shaped convolutional neural network (CNN) architectures [20, 24, 37, 41, 44, 62], notably UNet [44], UNet++ [62], UNet3+ [24], and nnU-Net [19], have become standard techniques for this purpose, achieving high-quality, high-resolution segmentation output.”
HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation (2022, background): “This structure has been further improved and utilized in different medical domains [8, 30, 21, 5].”
Dense Nested Attention Network for Infrared Small Target Detection (2021, background): “segmentation [26], [27], [28], [29] and hybrid attention in”
Channel prior convolutional attention for medical image segmentation (2023, methodology): “Comprehensive skip connections and deep supervision based on U-Net were utilized by (Huang et al., 2020) to e ff ectively leverage multi-scale information.”
UNeXt: MLP-based Rapid Medical Image Segmentation Network (2022, methodology): “Following UNet, a number of key extensions like UNet++ [29], UNet3+ [13], 3D UNet [7], V-Net [16], Y-Net [15] and KiUNet [21,22] have been proposed.”
MISSFormer: An Effective Transformer for 2D Medical Image Segmentation (2022, background): “According to this elegant U-shaped architecture, the variants of U-Net [4], [5], [6] achieved good performance and impressive results.”
Medical Transformer: Gated Axial-Attention for Medical Image Segmentation (2021, background): “Networks like U-Net [17], V-Net [15], 3D U-Net [4], Res-UNet [27], Dense-UNet [13], YNet [14], U-Net++ [31], KiU-Net [22,21] and U-Net3+ [8] have been proposed specifically for performing image and volumetric segmentation for various medical imaging modalities.”