High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks

By Zifeng Wu, Chunhua Shen, Anton van den Hengel. 2016

130 citations recorded by the source corpus.

We propose a method for high-performance semantic image segmentation (or semantic pixel labelling) based on very deep residual networks, which achieves the state-of-the-art performance. A few design factors are carefully considered to this end. We make the following contributions. (i) First, we evaluate different variations of a fully convolutional residual network so as to find the best configuration, including the number of layers, the resolution of feature maps, and the size of field-of-view. Our experiments show that further enlarging the field-of-view and increasing the resolution of feature maps are typically beneficial, which however inevitably leads to a higher demand for GPU memories. To walk around the limitation, we propose a new method to simulate a high resolution network with a low resolution network, which can be applied during training and/or testing. (ii) Second, we propose an online bootstrapping method for training. We demonstrate that online bootstrapping is critically important for achieving good accuracy. (iii) Third we apply the traditional dropout to some of the residual blocks, which further improves the performance. (iv) Finally, our method achieves the currently best mean intersection-over-union 78.3\% on the PASCAL VOC 2012 dataset, as well as on the recent dataset Cityscapes.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

The Reversible Residual Network: Backpropagation Without Storing Activations (2017, background): “Due to memory constraints, modern architectures are often trained with a mini-batch size of 1 (e.g. [32, 35]), which is inefﬁcient for stochastic gradient methods.”
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, neutral): “This has also been observed independently by [81].”
Universal Adversarial Perturbations Against Semantic Image Segmentation (2017, background): “To address this issue, we set the loss of target pixels which are predicted as the desired target with a confidence above τ to 0 [26].”
BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation (2018, methodology): “Meanwhile, we compute the loss function with the online bootstrapping strategy as described in [33].”
Loss Max-Pooling for Semantic Image Segmentation (2017, methodology): “We have experimentally validated the effectiveness of our new loss function and showed consistently improved results on standard benchmark datasets for semantic segmentation.”
LIDAR-based driving path generation using fully convolutional neural networks (2017, background): “In [10], the authors have shown that working with high-resolution feature maps is helpful for achieving better performance.”
Fast LIDAR-based road detection using fully convolutional neural networks (2017, background): “feature maps, two aspects that have been shown to improve segmentation accuracy [17].”
Recent progress in semantic image segmentation (2018, background): “Furthermore, paper (Wu et al. 2016b) studies the deep residual networks and explains some behaviors that have been observed experimentally.”
Adaptive Affinity Field for Semantic Segmentation (2018, methodology): “We base our implementations on ResNet101 [3], and we fine-tune models pre-trained on ImageNet [35] for both architectures on VOC and FCN [2] on Cityscapes.”
Estimating Depth From Monocular Images as Classification Using Deep Fully Convolutional Residual Networks (2016, methodology): “ence can be improved by other pixels that connected to it. An overview of our proposed depth estimation model is illustrated in Fig. 1. In practice, we also apply the online bootstrapping proposed in [9] to eliminate the pixels that are easy to label, and at the same time, to mine the hard training ”
ACFNet: Attentional Class Feature Network for Semantic Segmentation (2019, methodology): “Moreover, we apply the online bootstrapping [39] and multi-scale (MS), left-right flipping (Flip) to improve the performance based on the ResNet-101+ ASPP + ACF.”
ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation (2018, methodology): “Our proposal aims at solving an efficiency limitation that is inherently present in commonly adopted versions of the residual layer, which is used in several recent ConvNets that achieve top accuracy in classification [6] and segmentation tasks [8][20][11].”