ExFuse: Enhancing Feature Fusion for Semantic Segmentation

By Zhenli Zhang, Xiangyu Zhang, Chao Peng, Dazhi Cheng, Jian‐Yuan Sun. 2018

323 citations recorded by the source corpus.

Modern semantic segmentation frameworks usually combine low-level and high-level features from pre-trained backbone convolutional models to boost performance. In this paper, we first point out that a simple fusion of low-level and high-level features could be less effective because of the gap in semantic levels and spatial resolution. We find that introducing semantic information into low-level features and high-resolution details into high-level features is more effective for the later fusion. Based on this observation, we propose a new framework, named ExFuse, to bridge the gap between low-level and high-level features thus significantly improve the segmentation quality by 4.0\% in total. Furthermore, we evaluate our approach on the challenging PASCAL VOC 2012 segmentation benchmark and achieve 87.9\% mean IoU, which outperforms the previous state-of-the-art results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Quoc V. Le
2. Alan L. Yuille
3. Trevor Darrell
4. Barret Zoph
5. Mohit Bansal
6. Jianfeng Gao
7. Liang-Chieh Chen
8. Li Fei-Fei

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Alan L. Yuille
3. Dacheng Tao
4. Ming-Hsuan Yang
5. Xiaogang Wang
6. Trevor Darrell
7. Quoc V. Le
8. Jitendra Malik

In Radiology, Nuclear Medicine and Imaging

1. Ekin Dogus Cubuk
2. Barret Zoph
3. Quoc V. Le
4. Golnaz Ghiasi
5. Tsung-Yi Lin
6. Yin Cui
7. Alan L. Yuille
8. Chenxi Liu

Citations in the corpus

• Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, positive): “On both PASCAL detection and PASCAL semantic Model mIOU DeepLabv3+ † [4] 84.6 ExFuse † [70] 85.8 Eff-B7 [73] 85.2 Eff-L2 [73] 88.7 Eff-B7 NAS-FPN 83.9 w/ Copy-Paste pre-training (+2.7) 86.6 Table 6.”
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018, neutral): “ncoder-decoder: The encoder-decoder networks have been successfully applied to many computer vision tasks, including human pose estimation [53], object detection [54,55,56], and semantic segmentation [11,57,21,22,58,59,60,61,62,63,64]. Typically, the encoder-decoder networks contain (1) an encoder m”
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation (2019, positive): “In addition to adopting those networks as backbones for semantic segmentation, one could employ the encoder-decoder structures [64, 2, 55, 44, 60, 58, 33, 79, 18, 11, 87, 83] which ef-ficiently captures the long-range context information while keeping the detailed object boundaries.”
• Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, neutral): “In the recent years, we have witnessed substantial progress in this area [1], [2], [3], [4], [5], [6], mostly thanks to the introduction of fully convolutional networks [1] and to the integration of several other architectural components, such as attention models [7], [8], [9] or multi-scale fusion ”
• Dual Attention Network for Scene Segmentation (2018, neutral): “Different from recent work (Zhang et al. 2018b) which explores channel relationships by a global pooling or encoding layer, we exploit spatial information at all corresponding positions to model channel correlations.”
• Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “…heavy up-sample processes [141], recombinator networks [55]; improving skip connections with more or complicated convolutional units [64], [111], [180], as well as sending information from low-resolution skip connections to high-resolution skip connections [189] or exchanging information between…”
• Image Segmentation Using Deep Learning: A Survey (2020, neutral): “Fortunately, with the increasing popularity of deep learning models, the trend has been TABLE 1 Accuracies of Segmentation Models on the PASCALVOC Test Set Method Backbone mIoU FCN [30] VGG-16 62.2 CRF-RNN [38] - 72.0 CRF-RNN [38] - 74.7 BoxSup* [119] - 75.1 Piecewise [39] - 78.0 DPN [40] - 77.5 Dee”
• F3Net: Fusion, Feedback and Focus for Salient Object Detection (2019, unknown): “(Zhang et al. 2018a) designed a bi-directional message passing model for better feature selection and integration.”
• Asymmetric Non-Local Neural Networks for Semantic Segmentation (2019, unknown): “To evaluate our method, we carry out detailed experiments on three semantic segmentation datasets: Cityscapes [9], ADE20K [50] and PASCAL Context [21].”
• GFF: Gated Fully Fusion for Semantic Segmentation (2019, unknown): “For fair comparison, all methods are only trained using fine annotated dataset (Lin et al. 2017a) PSPNet (Zhao et al. 2017) PSANet (Zhao et al. 2018) EncNet (Zhang et al. 2018a) GCUNet (Li and Gupta 2018) Table 6: State-of-the-art comparison experiments on ADE20K validation set.”
• HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation (2020, unknown): “As previous research(Zhang et al. 2018) indicates, deep neural networks represent more semantic features as the layers going deeper.”