SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
By Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla. 2015
18,199 citations recorded by the source corpus.
We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1] . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dacheng Tao
- Yoshua Bengio
- Trevor Darrell
- Sergey Levine
- Quoc V. Le
- Bernt Schiele
- Mohit Bansal
- Luc Van Gool
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Yi Yang
- Alan L. Yuille
Citations in the corpus
- Panoptic Feature Pyramid Networks (2019, methodology): “As an alternative to dilation, an encoder-decoder [2] or ‘U-Net’ [49] architecture can be used to increase feature resolution [26, 44, 20, 47].”
- Beyond Skip Connections: Top-Down Modulation for Object Detection (2016, methodology): “Deconvolutional [36, 55] and encoder-decoder networks [3] are often used to generate such finer features by using the ‘unpool’ operation.”
- The Cityscapes Dataset for Semantic Urban Scene Understanding (2016, background): “Other work takes advantage of deep learning for explicitly integrating global scene context in the prediction 5 of pixel-wise semantic labels, in particular through CNNs [4,39,44,67] or Recurrent Neural Networks (RNNs) [8,52].”
- A U-Net Based Discriminator for Generative Adversarial Networks (2020, methodology): “Encoder-decoder networks [2, 39] constitute a powerful method for dense prediction.”
- The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation (2016, methodology): “VGG [29] or ResNet101 [11]) to improve their segmentation results [20, 1, 4].”
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, background): “size the encoder-decoder structure of [100], [102] may allevi-”
- ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes (2021, neutral): “from mIoU 57% in 2015 [4] to 84% in 2020 [15] on CityScapes [17], or from AP50 29.”
- Scene Parsing through ADE20K Dataset (2017, methodology): “Scene parsing As for baselines of scene parsing on SceneParse150 benchmark, we train three semantic segmentation networks: SegNet [1], FCN-8s [19], and DilatedNet [4, 34].”
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018, methodology): “On the other hand, encoder-decoder models [61, 3] lend themselves to faster computation (since no features are dilated) in the encoder path and gradually recover sharp object boundaries in the decoder path.”
- Rethinking Atrous Convolution for Semantic Image Segmentation (2017, methodology): “Deconvolutional layers (or transposed convolution) [79, 51, 54, 3, 61, 58] have been employed to recover the spatial resolution.”
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size (2016, methodology): “ImageNettrained DNNs have also been applied to a number of applications pertaining to autonomous driving, including pedestrian and vehicle detection in images [12, 7] and videos [2], as well as segmenting the shape of the road [1].”
- Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation (2019, methodology): “In addition to adopting those networks as backbones for semantic segmentation, one could employ the encoder-decoder structures [63, 2, 55, 44, 60, 58, 33, 78, 18, 11, 86, 82] which efficiently captures the long-range context information while keeping the detailed object boundaries.”
- Learning to Prune Filters in Convolutional Neural Networks (2018, methodology): “Accuracy drop FCN32s 63.7 55.4 37.0 49.1 1.3 (3.4) SegNet 56.9 63.9 42.4 53.0 -2.1 (3.0) ?”
- An overview of deep learning in medical imaging focusing on MRI (2018, methodology): “informatik.uni-freiburg. de/people/ronneber/u-net V-net [85] https://github.com/faustomilletari/ VNet SegNet: A Deep Convolutional EncoderDecoder Architecture for Robust Semantic Pixel-Wise Labelling [325] https://mi.eng.cam.ac.uk/projects/ segnet Brain lesion synthesis using GANs [213] https://gith”
- Deep Reinforcement Learning framework for Autonomous Driving (2017, background): “Although far from trivial, recognition is a relatively easy task nowadays thanks to advances in Deep Learning (DL) algorithms, which have reached human level recognition or above at several object detection and classification problems [8] [2].”
- Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation (2022, neutral): “For instance, to reconstruct high-resolution prediction images, UNet [38] and its variant [1] use an encoder-decoder structure to segment images.”