GFF: Gated Fully Fusion for Semantic Segmentation

By Xiangtai Li, Houlong Zhao, Lei Han, Yunhai Tong, Kuiyuan Yang. 2019

238 citations recorded by the source corpus.

Semantic segmentation generates comprehensive understanding of scenes through densely predicting the category for each pixel. High-level features from Deep Convolutional Neural Networks already demonstrate their effectiveness in semantic segmentation tasks, however the coarse resolution of high-level features often leads to inferior results for small/thin objects where detailed information is important. It is natural to consider importing low level features to compensate for the lost detailed information in high-level features. Unfortunately, simply combining multi-level features suffers from the semantic gap among them. In this paper, we propose a new architecture, named Gated Fully Fusion(GFF), to selectively fuse features from multiple levels using gates in a fully connected way. Specifically, features at each level are enhanced by higher-level features with stronger semantics and lower-level features with more details, and gates are used to control the propagation of useful information which significantly reduces the noises during fusion. We achieve the state of the art results on four challenging scene parsing datasets including Cityscapes, Pascal Context, COCO-stuff and ADE20K.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, neutral): “GFF-Net [42] 82.”
Scene Segmentation With Dual Relation-Aware Attention Network (2020, methodology): “Compared with the work [57], we adopt a multilevel feature to obtain a gating map and achieve higher performance.”
Changer: Feature Interaction is What You Need for Change Detection (2022, methodology): “Additionally, several attention-based methods make feature fusion more ﬂexible and learnable [1,11,14,22,49].”
SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation (2022, unknown): “Since FCN [53] was proposed, convolutional neural networks (CNNs) [1, 64, 86, 94, 19, 87, 71, 20, 45] have achieved great success and become a popular architecture for semantic segmentation.”
Improving Semantic Segmentation via Decoupled Body and Edge Supervision (2020, unknown): “AAF [44] ResNet-101 79.1 PSANet [56] ResNet-101 80.1 DFN [57] ResNet-101 79.3 DenseASPP [34] DenseNet-161 80.6 DAnet [10] ResNet-101 81.5 CCNet [11] ResNet-101 81.4 BAFNet [43] ResNet-101 81.4 ACFNet [58] ResNet-101 81.9 GFFnet [59] ResNet-101 82.”
RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentation (2021, methodology): “Inspired by the mature Gating mechanism[9, 29, 19], which can aggregate information adaptively by measuring the importance of each feature, our gated fusion module is designed based on the normal addition-based fusion by filtering information flow with gates.”
Transformer-Based Visual Segmentation: A Survey (2023, background): “…including better encoder-decoder frameworks [58], [59], larger kernels [60], [61], multiscale pooling [11], [62], multiscale feature fusion [12], [63], [64], [65], non-local modeling [18], [66], [67], efﬁcient modeling [68], [69], [70], and better boundary delineation [71], [72], [73], [74].”
Semantic Flow for Fast and Accurate Scene Parsing (2020, background): “A common strategy is to fuse multi-level feature maps for high-resolution spatiality and strong semantics [1,28,33,46,51].”