3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
By Özgün Çiçek, A. Abdulkadir, S. Lienkamp, Thomas Brox, Olaf Ronneberger. 2016
8,190 citations recorded by the source corpus.
This paper introduces a network for volumetric segmentation that learns from sparsely annotated volumetric images. We outline two attractive use cases of this method: (1) In a semi-automated setup, the user annotates some slices in the volume to be segmented. The network learns from these sparse annotations and provides a dense 3D segmentation. (2) In a fully-automated setup, we assume that a representative, sparsely annotated training set exists. Trained on this data set, the network densely segments new volumetric images. The proposed network extends the previous u-net architecture from Ronneberger et al. by replacing all 2D operations with their 3D counterparts. The implementation performs on-the-fly elastic deformations for efficient data augmentation during training. It is trained end-to-end from scratch, i.e., no pre-trained network is required. We test the performance of the proposed method on a complex, highly variable 3D structure, the Xenopus kidney, and achieve good results for both use cases.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Vladlen Koltun
- Chi-Wing Fu
- P. Heng
- Daniel Cohen-Or
- Maxim Tatarchenko
- Jiajun Wu
- Leonidas J. Guibas
- Lequan Yu
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Jitendra Malik
- Trevor Darrell
- Andrew Zisserman
- Yi Yang
In Model Architectures
- Ming-Ming Cheng
- P. Heng
- Vladlen Koltun
- Ming-Hsuan Yang
- Deng-Ping Fan
- Hao Chen
- Fahad Shahbaz Khan
- Xinggang Wang
Citations in the corpus
- V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation (2016, methodology): “More recently, this approach was extended to 3D and applied to segmentation of volumetric data acquired from a confocal microscope [3].”
- A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, methodology): “Amongst successful deep learning models are [2], [5], using U-Net for medical segmentation.”
- Visual Feature Attribution Using Wasserstein GANs (2017, methodology): “As in the original 3D U-Net [10] we use batch normalisation for all layers except the final one.”
- NAS-Unet: Neural Architecture Search for Medical Image Segmentation (2019, background): “[29] first proposed a 3D U-Net network architecture that implements 3D image segmentation by inputting a continuous 2D slice sequence of 3D images.”
- RealPoint3D: Point Cloud Generation from a Single Image with Complex Background (2018, methodology): “All rights reserved. achieved great success for object classification, detection and segmentation (Cicek et al. 2016; Chen et al. 2016; Meagher 1980; Wu et al. 2016; Choy et al. 2016; Mat-urana and Scherer 2015; Tatarchenko, Dosovitskiy, and Brox 2017).”
- Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs (2017, methodology): “Applications include single- and multi-view 3D object reconstruction trained in supervised [14, 5, 7, 17] and unsupervised [37, 12, 19] ways, probabilistic generative modeling of 3D shapes [36, 35, 29], semantic segmentation [2, 4] and shape deformation [39].”
- A survey on deep learning in medical image analysis (2017, methodology): “A similar approach was used by Çiçek et al. (2016) for 3D data.”
- Deep Learning for Medical Image Analysis (2017, neutral): “ided diagnosis, image registration and multi-modal image analysis, image segmentation and retrieval. There has been some application that using deep learning in medical application like cell tracking [1] and organ cancer detection. Doctors use magnetic resonance images as eective tools to diagnosis ”
- Image Segmentation Using Deep Learning: A Survey (2020, background): “[50] proposed a U-Net architecture for 3D images.”
- Transformers in Medical Imaging: A Survey (2022, background): “[150] also propose a hybrid architecture consisting of transformer layers in the decoder part of 3D UNet [151] to effectively segment tumors from volumetric breast data.”
- On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task (2017, methodology): “72 3D U-net[3]-dice Feature forwarding 19.”
- Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning (2017, background): “In contrast, P-Net performs noticeably better than FCN and U-Net when dealing with these two unseen objects.”
- JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation (2020, background): “It is further extended to 3D U-Net [59], TernausNet [60], and U-Net ++ [61] with promising performance on versatile image segmentation tasks.”
- Modified U-Net (mU-Net) with Incorporation of Object-Dependent High Level Features for Improved Liver and Liver-Tumor Segmentation in CT Images (2019, background): “So, the U-Net has become the most popular base network widely used in biomedical image segmentation [25]–[27].”