Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks

By Jie Hu, Li Shen, Samuel Albanie, Gang Sun, Andrea Vedaldi. 2018

700 citations recorded by the source corpus.

While the use of bottom-up local operators in convolutional neural networks (CNNs) matches well some of the statistics of natural images, it may also prevent such models from capturing contextual long-range feature interactions. In this work, we propose a simple, lightweight approach for better context exploitation in CNNs. We do so by introducing a pair of operators: gather, which efficiently aggregates feature responses from a large spatial extent, and excite, which redistributes the pooled information to local features. The operators are cheap, both in terms of number of added parameters and computational complexity, and can be integrated directly in existing architectures to improve their performance. Experiments on several datasets show that gather-excite can bring benefits comparable to increasing the depth of a CNN at a fraction of the cost. For example, we find ResNet-50 with gather-excite operators is able to outperform its 101-layer counterpart on ImageNet with no additional learnable parameters. We also propose a parametric gather-excite operator pair which yields further performance gains, relate it to the recently-introduced Squeeze-and-Excitation Networks, and analyse the effects of these changes to the CNN feature activation statistics.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Visual Transformers: Token-based Image Representation and Processing for Computer Vision (2020, methodology): “Attention was first used to modulate the feature map: some attention values are computed from the input and multiplied to the feature map as in [9, 7, 8, 10].”
Exploring Self-Attention for Image Recognition (2020, background): “In channelwise attention models [34, 15, 14], attention weights reweight activations in different channels.”
Stand-Alone Self-Attention in Vision Models (2019, methodology): “Likewise, spatially-aware attention mechanisms have been used to augment CNN architectures to provide contextual information for improving object detection [32] and image classification [33–35].”
Attention Augmented Convolutional Networks (2019, methodology): “We compare Attention Augmentation (AA) against other forms of attention including Squeeze-and-Excitation (SE) [17] and the parameter-free formulation of Gather-Excite (GE) [16].”
Natural Adversarial Examples (2019, background): “Convolutional neural networks with self-attention [32] are designed to better capture long-range dependencies and interactions across an image.”
VOLO: Vision Outlooker for Visual Recognition (2021, neutral): “vision task [22], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41].”
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “Recently, methods adding noises to the internal features of CNNs [17, 8, 45] or adding extra path to the architecture [15, 14] have been proposed to enhance image classification performance.”
ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context (2020, positive): “This is consistent with the observation in a similar study of SE on image classiﬁcation models [34].”
Dynamic Neural Networks: A Survey (2021, neutral): “in one framework [21], [129], [130], [131] (see Fig.”
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet (2021, methodology): “Self-attention in CNNs Self-attention mechanism has been widely applied to CNNs in vision task [38, 57, 19, 47, 20, 39, 1, 6, 18, 31, 42, 13, 50, 48].”
LambdaNetworks: Modeling Long-Range Interactions Without Attention (2021, background): “Closest to our work are channel and linear attention mechanisms (Hu et al., 2018c; Katharopoulos et al., 2020; Choromanski et al., 2020).”
ConViT: improving vision transformers with soft convolutional inductive biases (2021, neutral): “A breadth of approaches have been taken to imbue CNN architectures with nonlocality [34, 35, 36, 28].”
SPGNet: Semantic Prediction Guidance for Scene Parsing (2019, methodology): “Recently, SENets [25] and GENets [24] employ the ‘squeeze-and-excite’ (Figure 1 (a)) or more general ‘gather-and-excite’ framework (Figure 1 (b)) and obtain remarkable results on image classification task.”
Conditional DETR for Fast Training Convergence (2021, methodology): “SENet [14], GENet [13] abd Lite-HRNet [45] learn from the input the channel-wise weights.”
FSPool: Learning Set Representations with Featurewise Sort Pooling (2019, methodology): “The weight vector allows different weightings of different ranks and is similar in spirit to the parametric version of the gather step in Gather-Excite [13].”
Context-aware Feature Generation For Zero-shot Semantic Segmentation (2020, methodology): “Inspired by [13], we element-multiply Zn to Fn after applying sigmoid activation (denoted as φ) as residual attention, that is, our CM outputs the new feature map Xn = Fn + Fn ⊙ φ (Zn) ∈ Rh×w×l as both the target of feature generation and the input of classifierC .”
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention (2024, neutral): “Existing plug-and-play attention methods can be primarily categorized into three types: channel attention [23, 27, 62, 55, 31, 41], spatial attention [22, 56, 42, 9], and hybrid channel-spatial attention [57, 40, 38, 14, 20, 65, 39, 61].”
Conformer: Local Features Coupling Global Representations for Visual Recognition (2021, methodology): “To alleviate such a limitation, one solution is to define larger receptive fields by introducing deeper architectures and/or more pooling operations [22, 21].”
Visual attention network (2022, methodology): “The second one is to use large kernel convolution [85,78,32,57] to build relevance and produce attention map.”
Multi-Stage Progressive Image Restoration (2021, background): “Driven by its success in high-level tasks such as image classification [31, 32, 79], segmentation [21, 35] and detection [74, 79], attention modules have been used in low-level vision tasks [38].”