ACFNet: Attentional Class Feature Network for Semantic Segmentation

By Fan Zhang, Yanqin Chen, Zhihang Li, Zhibin Hong, Jingtuo Liu, Feifei Ma, Junyu Han, Errui Ding. IEEE International Conference on Computer Vision, 2019

301 citations recorded by the source corpus.

Recent works have made great progress in semantic segmentation by exploiting richer context, most of which are designed from a spatial perspective. In contrast to previous works, we present the concept of class center which extracts the global context from a categorical perspective. This class-level context describes the overall representation of each class in an image. We further propose a novel module, named Attentional Class Feature (ACF) module, to calculate and adaptively combine different class centers according to each pixel. Based on the ACF module, we introduce a coarse-to-fine segmentation network, called Attentional Class Feature Network (ACFNet), which can be composed of an ACF module and any off-the-shell segmentation network (base network). In this paper, we use two types of base networks to evaluate the effectiveness of ACFNet. We achieve new state-of-the-art performance of 81.85% mIoU on Cityscapes dataset with only finely annotated data used for training.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/289976cbe82222a30a388b4af380244cb39360d0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers (2022, methodology): “Additional FCNbased methods improve dense semantic segmentation by refining context priors [43], [44], [45], [46], [47], [48], [49], enhancing boundary cues [50], [51], [52], and appending various attention modules [53], [54], [55], [56], [57] to boost the performance.”
SAM-Adapter: Adapting Segment Anything in Underperformed Scenes (2023, background): “…the remarkable advancements in deep-learning-based methods such as fully convolutional networks (FCN) [35], encoder-decoder structures [41, 50, 14, 1, 29], dilated convolutions [3, 4, 5, 6, 34], pyramid structures [60, 5, 52, 6, 57], attention modules [51, 16, 62], and transformers [54, 48, 45, 8].”
Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, background): “Therefore, substantial researches have been implemented which aim to alleviate the bottleneck to efficiency and push the boundaries of attention, including accelerating the generation process of the attention matrix [47, 74-76], pruning the structure of the attention block [77], and optimizing atten”
Hierarchical Multi-Scale Attention for Semantic Segmentation (2020, background): “OCRNet [9], DANET [10], CFNet [11], OCNet [12] and other related work [13, 14, 15, 16, 17, 18, 19, 20] use such relationships to build better context.”
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation (2020, unknown): “Besides exploiting various contextual information from the visual cues [25,11,14,12,5,43], depth data have recently been utilized as supplementary information to RGB data to achieve improved segmentation accuracy [26,33,44,4,15,23,6,19].”
A Survey on Vision Transformer (2020, background): “PSANet [151], OCNet [139], DANet [38] and CFNet [147] are the first works that introduce the self-attention module to semantic segmentation task, which consider and augment the relation and similarity [146, 74, 46, 89, 130] between the contextual pixels.”
Learning Statistical Texture for Semantic Segmentation (2021, background): “Most of the modern semantic segmentation models [23, 13, 25, 19, 5, 32, 8, 12] are based on fully convolutional networks (FCN) [23], which first replaces the fully connected layers in common classification networks by convolutional layers, getting pixel-level prediction results.”
Improving Semantic Segmentation via Decoupled Body and Edge Supervision (2020, unknown): “ACFNet [58] ResNet-101 81.”
Object-Contextual Representations for Semantic Segmentation (2019, methodology): “For example, the region formed in Double Attention do not correspond to an object class, and the relation in ACFNet [74] is computed only from the pixel representation w/o using the object region representation.”
OCNet: Object Context for Semantic Segmentation (2021, methodology): “There also exist some con-current efforts (Fu et al. 2019a; Zhang et al. 2019b; Huang et al. 2019; Li et al. 2019; Zhao et al. 2018) that exploit the semantic relations between pixels to construct the context, and our approach is different from most of them as we propose a simple yet effective…”
Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (2021, background): “Recent breakthroughs in semantic segmentation have mainly come from multi-scale feature aggregation [43, 2, 35, 10] or attention mechanisms [7, 38, 15, 28, 3, 44, 41].”
Disentangled Non-Local Neural Networks (2020, unknown): “7 ACFNet [40] ResNet-101 √ 81.”
Interactive Two-Stream Decoder for Accurate and Fast Saliency Detection (2020, methodology): “In the U-shape network, features from the encoder are skip-connected to the features with the same spatial sizes in the decoder, where this simple connection has been demonstrated its power in various pixel-wise predictions [14, 39, 43].”
Real-Time Scene Text Detection With Differentiable Binarization and Adaptive Scale Fusion (2022, methodology): “ANN [67], CCNet [20], GCNet [3], DANet [11] and ACFNet [62] all used self-attention to fuse different features for contextual information.”
Visual Attention Methods in Deep Learning: An In-Depth Survey (2022, background): “[172] introduced ACFNet, a novel idea to exploit the contextual”