GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond

By Jiarui Xu, Stephen Lin, Fangyun Wei, Han Hu. 2019

1,934 citations recorded by the source corpus.

The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies, via aggregating query-specific global context to each query position. However, through a rigorous empirical analysis, we have found that the global contexts modeled by non-local network are almost the same for different query positions within an image. In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation. We further observe that this simplified design shares similar structure with Squeeze-Excitation Network (SENet). Hence we unify them into a three-step general framework for global context modeling. Within the general framework, we design a better instantiation, called the global context (GC) block, which is lightweight and can effectively model the global context. The lightweight property allows us to apply it for multiple layers in a backbone network to construct a global context network (GCNet), which generally outperforms both simplified NLNet and SENet on major benchmarks for various recognition tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Dacheng Tao
3. Jiashi Feng
4. Alan L. Yuille
5. Lu Yuan
6. Luc Van Gool
7. Jianwei Yang
8. Quoc V. Le

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Ming-Hsuan Yang
4. Xiaogang Wang
5. Alan L. Yuille
6. Chunhua Shen
7. Yi Yang
8. Jiashi Feng

In Model Architectures

1. Jianfeng Gao
2. Jianwei Yang
3. Lu Yuan
4. Jiashi Feng
5. Alan L. Yuille
6. Bin Xiao
7. Dacheng Tao
8. Xiyang Dai

Citations in the corpus

• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, methodology): “The selfattention layers can complement backbones [61, 7, 3, 65, 21, 68, 51] or head networks [29, 24] by providing the capability to encode distant dependencies or heterogeneous interactions.”
• Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions (2021, methodology): “Method #Param (M) GFLOPs Mask R-CNN 1x APm AP50 AP m 75 ResNet50+GC r4 [1] 54.”
• MMDetection: Open MMLab Detection Toolbox and Benchmark (2019, background): “• GCNet [3]: global context block that can efficiently model the global context, proposed in 2019.”
• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers (2020, neutral): “…75.52 FCN (80k, MS) [39] ResNet-101 76.61 PSPNet [60] ResNet-101 78.50 DeepLab-v3 [10] (MS) ResNet-101 79.30 NonLocal [48] ResNet-101 79.10 CCNet [25] ResNet-101 80.20 GCNet [4] ResNet-101 78.10 Axial-DeepLab-XL [47] additionally pixel-wise accuracy is reported following the existing practice.”
• Bottleneck Transformers for Visual Recognition (2021, methodology): “[3,4,6,6], refers to the use of 3, 4, 6 and 6 blocks respectively in stages c2,c3,c4,c5.”
• Efficient Attention: Attention with Linear Complexities (2018, positive): “GCNet [3] proposes to replace the global average pooling by an adaptive pooling layer, which uses a linear layer to compute the weight for each position.”
• Focal Self-attention for Local-Global Interactions in Vision Transformers (2021, neutral): “This is also supported by the recent advanced CNN models [36, 66, 64, 71, 2, 8, 52], which showed that global attention or interaction can effectively improve the performance.”
• GaitPart: Temporal Part-Based Model for Gait Recognition (2020, methodology): “…way, the sequence of micro-motion feature vectors, S m , will be obtained, and can be formulated as Further, in order to obtain the more discriminative representations for micro-motion, the channel-wise attention mechanism is introduced to re-weight the feature vector at each moment[9, 24, 12, 4].”
• GPS-Net: Graph Property Sensing Network for Scene Graph Generation (2020, methodology): “However, as pointed out in [21], these methods [22, 23, 24] may fail to learn object-specific context due to the utilization of the first-order linear model.”
• FcaNet: Frequency Channel Attention Networks (2020, methodology): “To improve efficiency, GCNet [4] proposes to use a simple spatial attention module and replace the original spatial downsampling process.”
• Involution: Inverting the Inherence of Convolution for Visual Recognition (2021, methodology): “like NLNet [52], CCNet [22], and GCNet [4].”
• Neural Architecture Search for Lightweight Non-Local Networks (2020, methodology): “[4] compute the affinity matrix via a convolutional layer; Bello et al .”
• Conformer: Local Features Coupling Global Representations for Visual Recognition (2021, background): “The other solution is the global attention mechanism [45, 7, 4, 20, 38], which has demonstrated great advantage in capturing long-distance dependencies in natural language processing [42, 15, 5].”
• Anticipative Video Transformer (2021, background): “The general idea of self-attention in vision dates back to non-local means [9], and is incorporated into contemporary network architectures as non-local blocks [10, 54, 90, 92] and gating mechanisms [30, 45, 60, 94].”