nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation

By Fabian Isensee, Jens Petersen, André Klein, David Zimmerer, Paul F. Jaeger, Simon A. A. Kohl, Jakob Wasserthal, Gregor Koehler, T. Norajitra, Sebastian J. Wirkert and 1 more. 2018

1,100 citations recorded by the source corpus.

The U-Net was presented in 2015. With its straight-forward and successful architecture it quickly evolved to a commonly used benchmark in medical image segmentation. The adaptation of the U-Net to novel problems, however, comprises several degrees of freedom regarding the exact architecture, preprocessing, training and inference. These choices are not independent of each other and substantially impact the overall performance. The present paper introduces the nnU-Net ('no-new-Net'), which refers to a robust and self-adapting framework on the basis of 2D and 3D vanilla U-Nets. We argue the strong case for taking away superfluous bells and whistles of many proposed network designs and instead focus on the remaining aspects that make out the performance and generalizability of a method. We evaluate the nnU-Net in the context of the Medical Segmentation Decathlon challenge, which measures segmentation performance in ten disciplines comprising distinct entities, image modalities, image geometries and dataset sizes, with no manual adjustments between datasets allowed. At the time of manuscript submission, nnU-Net achieves the highest mean dice scores across all classes and seven phase 1 tasks (except class 1 in BrainTumour) in the online leaderboard of the challenge.

Subfields: Computer Vision and Pattern Recognition, Neurology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Neurology

Citations in the corpus

Resource Optimized Neural Architecture Search for 3D Medical Image Segmentation (2019, methodology): “Most researchers manually determine the best 2D or 3D input patch size, type of down sampling operations along with their stride, and skip connection methods, taking into account available GPU memory and receptive ﬁeld size [3,4,5].”
Transformers in Medical Imaging: A Survey (2022, methodology): “Furthermore, BiTr-UNet adopts a post-processing strategy to eliminate a volume of predicted segmentation if the volume is smaller than a threshold [141] followed by model ensemble via majority voting [142].”
Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection (2018, methodology): “…pixel-wise cross entropy loss L CE , a soft Dice loss is applied, which has been shown to stabilize training on highly class imbalanced segmentation tasks e.g. in the medical domain [12]: where u is the softmax output of the network and v is a one hot encoding of the ground truth segmentation map.”
nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation (2024, background): “The introduction of nnU-Net in 2018 was a pivotal moment, highlighting that careful implementation and configuration of the architecture are more crucial for achieving state-of-the-art results than modifying the architecture itself [23,21].”
Deep learning-enabled medical computer vision (2021, background): “Similarly, the segmentation of objects has substantially improved 17,18 , particularly in challenging scenarios such as the biomedical segmentation of multiple types of overlapping cells in microscopy.”
Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19 (2020, methodology): “In the literature, there have been numerous techniques for lung segmentation with different purposes [63-67].”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, background): “Deep learning is now powering the majority of computer vision applications ranging from autonomous driving [93, 73] and medical imaging [78, 38] to image editing [69, 15, 98, 88, 70, 74].”
Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels (2019, methodology): “In the HighResNet networks (Section 4.1 - Medical imaging network), we used the RMSE loss for the regression task and the Dice + Cross-Entropy loss [39] for the segmentation task.”
Automated Design of Deep Learning Methods for Biomedical Image Segmentation (2019, methodology): “residual connections [26, 10], dense connections [18, 15], attention mechanisms [30] or dilated convolutions [5, 24]) represent a necessary condition for good performance on the KiTS task.”
A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images (2020, background): “Attention U-Net and ScSE U-Net achieved lower HD 95 values than 2D nnU-Net, but their Dice and RVE values were worse than those of 2D nnU-Net, demonstrating that it is difﬁcult to beat the nnU-Net structure, as also shown by previous works [37], [41].”
Medical Image Segmentation Using Deep Learning: A Survey (2020, background): “[136] argued that too much manual adjustment on network structure could lead to over-fitting for a given dataset, and therefore proposed a medical image segmentation framework no-newUNet (nnU-Net) that adapts itself to any new dataset.”