U-Net: Convolutional Networks for Biomedical Image Segmentation
By Olaf Ronneberger, Philipp Fischer, Thomas Brox. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015
98,692 citations recorded by the source corpus.
There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .
Subfields: Biophysics, Computer Vision and Pattern Recognition, Media Technology, Model Architectures
View Paper: https://arxiv.org/abs/1505.04597
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biophysics
- Richard Zhang
- Yang Song
- Alexei A. Efros
- Jun-Yan Zhu
- Vladlen Koltun
- Jonathan Ho
- Stefano Ermon
- Jong Chul Ye
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Andrew Zisserman
- Andrea Vedaldi
In Media Technology
- Luc Van Gool
- Radu Timofte
- Ming-Hsuan Yang
- Lei Zhang
- Chen Change Loy
- Chao Dong
- Wangmeng Zuo
- Kai Zhang
In Model Architectures
- Quoc V. Le
- Alan L. Yuille
- Yoshua Bengio
- Chunhua Shen
- Vladlen Koltun
- Jiashi Feng
- Luc Van Gool
- Kaiming He
Citations in the corpus
- Fully convolutional networks for semantic segmentation (2014, background): “es features for fusion and captures context with global pooling. The “deconvolutional network” approach of [42] restores resolution by proposals, stacks of learned deconvolution, and unpooling. U-Net [43] combines skip layers and learned deconvolution for pixel labeling of microscopy images. The dil”
- Denoising Diffusion Probabilistic Models (2020, methodology): “To represent the reverse process, we use a U-Net backbone similar to an unmasked PixelCNN++ [49, 45] with group normalization throughout [62].”
- High-Resolution Image Synthesis with Latent Diffusion Models (2021, methodology): “The neural backbone θ(◦, t) of our model is realized as a time-conditional UNet [66].”
- CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “Unet [28], Deeplabv3+ [9], RefineNet [21] and DFN [37] adopted encoder-decoder structures that fuse the information in lowlevel and high-level layers to predict segmentation mask.”
- Contrastive Multiview Coding (2019, methodology): “A UNet style architecture [63] is utilized to perform the segmentation task.”
- Making Convolutional Networks Shift-Invariant Again (2019, methodology): “The U-Net also uses upsampling layers, without any smoothing.”
- Feature Pyramid Networks for Object Detection (2016, methodology): “rarchy without combining features or scores. There are recent methods exploiting lateral/skip connections that associate low-level feature maps across resolutions and semantic levels, including U-Net [31] and SharpMask [28] for segmentation, Recombinator networks [17] for face detection, and Stacked”
- Panoptic Feature Pyramid Networks (2019, methodology): “In contrast to ‘symmetric’ decoders [49], FPN uses a lightweight decoder (see Fig.”
- Recurrent Neural Networks for Semantic Instance Segmentation (2017, background): “[27, 37], but are rarely used for the related task of semantic instance segmentation.”
- V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation (2016, methodology): “In [11], three fully convolutional deep neural networks, pre-trained on a classification task, were refined to produce segmentations while in [17], a brand new CNN model, especially tailored to tackle biomedical image analysis problems in 2D, was proposed.”
- Beyond Skip Connections: Top-Down Modulation for Object Detection (2016, methodology): “Conv-deconv [36] and encoder-decoder style networks [3, 42] have been used for image segmentation to utilize finer features via lateral connections.”
- Analyzing and Improving the Image Quality of StyleGAN (2019, positive): “Skip connections [29, 19], residual networks [15, 14, 26], and hierarchical methods [6, 41, 42] have proven highly successful also in the context of generative methods.”
- Generative Modeling by Estimating Gradients of the Data Distribution (2019, methodology): “In the experiments, our model s θ ( x , σ ) combines the architecture design of U-Net [39] with dilated/atrous convolution [53, 54, 6]—both of which have been proved very successful in semantic segmentation.”
- Projected GANs Converge Faster (2021, methodology): “However, our CSM block is simpler than a vanilla U-Net [65]: we only use a single convolutional layer at each scale.”
- Resolution-robust Large Mask Inpainting with Fourier Convolutions (2021, background): “Another popular group of choices for the completion network is architectures based on U-Net [37], such as [26, 50, 59, 27].”
- Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, background): “[67]) the latent does not explicitly share the structure (spatial or otherwise) of the inputs.”
- Cascaded Diffusion Models for High Fidelity Image Generation (2021, background): “Architectures The current best architectures for image diffusion models are U-Nets [19, 23], which are a natural choice to map corrupted data xt to reverse process parameters (µθ,Σθ) that have the same spatial dimensions as xt. Scalar conditioning, such as a class label or a diffusion timestep t, is”
- Diffusion Models Beat GANs on Image Synthesis (2021, methodology): “[31] adopted the UNet architecture [58] for diffusion models, which Jolicoeur-Martineau et al.”