Context Contrasted Feature and Gated Multi-scale Aggregation for Scene Segmentation

By Henghui Ding, Xudong Jiang, Bing Shuai, Ai Qun Liu, G. Wang. 2018

391 citations recorded by the source corpus.

Scene segmentation is a challenging task as it need label every pixel in the image. It is crucial to exploit discriminative context and aggregate multi-scale features to achieve better segmentation. In this paper, we first propose a novel context contrasted local feature that not only leverages the informative context but also spotlights the local information in contrast to the context. The proposed context contrasted local feature greatly improves the parsing performance, especially for inconspicuous objects and background stuff. Furthermore, we propose a scheme of gated sum to selectively aggregate multi-scale features for each spatial position. The gates in this scheme control the information flow of different scale features. Their values are generated from the testing image by the proposed network learnt from the training data so that they are adaptive not only to the training data, but also to the specific testing image. Without bells and whistles, the proposed approach achieves the state-of-the-arts consistently on the three popular scene segmentation datasets, Pascal Context, SUN-RGBD and COCO Stuff.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CCNet: Criss-Cross Attention for Semantic Segmentation (2018, background): “[41] further introduced PSPNet with pyramid pooling module to capture contextual information.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, methodology): “Some methods [14,22,32,37,38,50,55] is inspired by Non-local [85], utilizing attention mechanisms to establish connections between image contexts.”
Dual Attention Network for Scene Segmentation (2018, methodology): “Following [6, 10], we report our results on 171 categories including 80 objects and 91 stuff annotated to each pixel.”
Deep High-Resolution Representation Learning for Visual Recognition (2019, methodology): “We follow the standard testing procedure [32], [172].”
Strip Pooling: Rethinking Spatial Pooling for Scene Parsing (2020, background): “Besides, aggregating non-local context [36, 58, 29, 15, 7, 45, 21, 53, 66, 23, 14] is also effective for scene parsing.”
OCNet: Object Context Network for Scene Parsing (2018, neutral): “8 CCL (Ding et al., 2018) ResNet-101 51.”
Image Segmentation Using Deep Learning: A Survey (2020, methodology): “Other models use multiscale analysis for segmentation, among them Dynamic Multiscale Filters Network (DMNet) [56], Context Contrasted Network and gated multiscale aggregation (CCN) [57], Adaptive Pyramid Context Network (APC-Net) [58], MultiScale Context Intertwining (MSCI) [59], and salient object ”
GFF: Gated Fully Fusion for Semantic Segmentation (2019, background): “Comparison with Other Gate module : The work (Ding et al. 2018) also used gates for information control between adjacent layers.”
Transformer-Based Visual Segmentation: A Survey (2023, background): “During the last ten years, deep neural networks, Convolution Neural Networks (CNNs) [6], [7], [8], such as Fully Convolutional Networks (FCNs) [9], [10], [11], [12] have achieved remarkable successes for different segmentation tasks and led to much better results.”
Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation (2020, background): “[29], [30] propose a scheme of gated sum, which can control the information ﬂow of different scale in the encoder-decoder architecture.”
SegFix: Model-Agnostic Boundary Refinement for Segmentation (2020, neutral): “Other studies [5,52,17,41,40,30,18,19] also ex-”
Camouflaged Object Segmentation with Distraction Mining (2021, background): “Large-ﬁeld contextual features are captured in [42, 38], direction-aware contexts are explored in [20], and contextual contrasted features are leveraged in [8, 59].”
Survey on semantic segmentation using deep learning techniques (2019, background): “Recently, Ding et al. [115] proposed a context contrasted local (CCL) model to obtain multi-scale features (both context and local).”