MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation

By Nabil Ibtehaz, Mohammad Sohel Rahman. Neural Networks, 2019

2,084 citations recorded by the source corpus.

In recent years Deep Learning has brought about a breakthrough in Medical Image Segmentation. In this regard, U-Net has been the most popular architecture in the medical imaging community. Despite outstanding overall performance in segmenting multimodal medical images, through extensive experimentations on some challenging datasets, we demonstrate that the classical U-Net architecture seems to be lacking in certain aspects. Therefore, we propose some modifications to improve upon the already state-of-the-art U-Net model. Following these modifications, we develop a novel architecture, MultiResUNet, as the potential successor to the U-Net architecture. We have tested and compared MultiResUNet with the classical U-Net on a vast repertoire of multimodal medical images. Although only slight improvements in the cases of ideal images are noticed, remarkable gains in performance have been attained for the challenging ones. We have evaluated our model on five different datasets, each with their own unique challenges, and have obtained a relative improvement in performance of 10.15%, 5.07%, 2.63%, 1.41%, and 0.62% respectively. We have also discussed and highlighted some qualitatively superior aspects of MultiResUNet over classical U-Net that are not really reflected in the quantitative measures.

Subfields: Artificial Intelligence, Neurology, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/5bc8d8e541045d5c5a2b4c329e5fea5b5e3bc10d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Neurology

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing (2019, background): “In scenarios where abundant, high-quality training samples are unavailable, such as medical imaging [4], [5] and 3D reconstruction [6], the performance of deep networks may degrade significantly, and the results may even be worse than those from traditional approaches.”
DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation (2020, methodology): “7%, and Multi-ResUNet [17] by an approximate margin of 1.”
Medical Image Segmentation Review: The Success of U-Net (2022, background): “2) Multi-Resolution Blocks: To tackle the difﬁculty of analyzing objects at different scales, Ibtehaz et al. [13] introduce the MultiResUNet with inception-like blocks as a backbone.”
U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications (2020, methodology): “Images have been collected from papers in this survey, including: (a) Retinal vessel segmentation [107]; (b) Brain tumor detection and segmentation [108]; (c) Multi-organ abdominal segmentation (liver; spleen; left and right kidneys; pancreas; gallbladder; aorta; and inferior vena cava) on CT scans ”
Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation (2024, neutral): “…U-Net with advanced network blocks techniques such as dense connections [12], residual blocks [10], attention mechanisms [30], depthwise convolutions [11], and atrous convolutions [33,35], resulting in various modified UNet in CT, MRI, Ultrasound medical image segmentation [23,13,15,29,34,36].”
TransMed: Transformers Advance Multi-Modal Medical Image Classification (2021, background): “Multi-modal medical analysis is one of the most fundamental and challenging parts of medical image analysis [8].”
UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer (2021, background): “The other approach focuses on strengthening the skip connections by introducing additional non-linear transformations on the features propagating from the encoder stage, which should account for or somewhat balance the possible semantic gaps (Rahman 2020).”
Sharp U-Net: Depthwise Convolutional Network for Biomedical Image Segmentation (2021, methodology): “Inspired by Inception modules that are used in CNNs to allow for more efficient computation, Ibtehaz and Rahman [19] introduce MultiResUNet, an enhanced UNet architecture, which uses a chain of convolutional layers with residual connections instead of simply concatenating the feature maps from the e”
Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey (2020, methodology): “Recently, in [92], the authors have used U-Net with multiRes block for multimodal biomedical image segmentation and got better result than using classical U-Net.”
Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, methodology): “Meanwhile, the structural development based on residual connections is also a promising direction [25, 66-69].”
Medical Image Segmentation Using Deep Learning: A Survey (2020, methodology): “[43]proposed MultiResUNet including the Residual Path (ResPath), which makes the encoder features perform some additional convolution operations before fusing with the corresponding features in the decoder.”
SemiCDNet: A Semisupervised Convolutional Neural Network for Change Detection in High Resolution Remote-Sensing Images (2020, methodology): “To overcome the semantic gaps between encoder and decoder parts, many extensions of UNet have been proposed, such as MultiResUnet [54], MDU-UNet [55], and UNet ++ [49], which have opened up new perspectives of improving semantic segmentation performance for RS community.”
MA-Net: A Multi-Scale Attention Network for Liver and Tumor Segmentation (2020, methodology): “Many methods are proposed with applying skip connections to obtain Multi-scale information of images [7], [16], [18], [19].”
A Review of Deep-Learning-Based Medical Image Segmentation Methods (2021, neutral): “[48] proposed MultiResUNet that based on probable scopes for improvement to analyze the U-Net model architecture.”