SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

By Qing-Long Zhang, Yubin Yang. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2021

777 citations recorded by the source corpus.

Attention mechanisms, which enable a neural network to accurately focus on all the relevant elements of the input, have become an essential component to improve the performance of deep neural networks. There are mainly two attention mechanisms widely used in computer vision studies, spatial attention and channel attention, which aim to capture the pixel-level pairwise relationship and channel dependency, respectively. Although fusing them together may achieve better performance than their individual implementations, it will inevitably increase the computational overhead. In this paper, we propose an efficient Shuffle Attention (SA) module to address this issue, which adopts Shuffle Units to combine two types of attention mechanisms effectively. Specifically, SA first groups channel dimensions into multiple sub-features before processing them in parallel. Then, for each sub-feature, SA utilizes a Shuffle Unit to depict feature dependencies in both spatial and channel dimensions. After that, all sub-features are aggregated and a "channel shuffle" operator is adopted to enable information communication between different sub-features. The proposed SA module is efficient yet effective, e.g., the parameters and computations of SA against the backbone ResNet50 are 300 vs. 25.56M and 2.76e-3 GFLOPs vs. 4.12 GFLOPs, respectively, and the performance boost is more than 1.34% in terms of Top-1 accuracy. Extensive experimental results on common-used benchmarks, including ImageNet-1k for classification, MS COCO for object detection, and instance segmentation, demonstrate that the proposed SA outperforms the current SOTA methods significantly by achieving higher accuracy while having lower model complexity.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a6afad3cc55ce0876dba39663f20edcac6257d18

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Efficient Multi-Scale Attention Module with Cross-Spatial Learning (2023, background): “With the parallel substructures, Shuffle attention (SA) [14] grouped channel dimensions into multiple subfeatures and addressed them in parallel, which can be efficiently parallelized across multiple processors.”
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention (2024, neutral): “Existing plug-and-play attention methods can be primarily categorized into three types: channel attention [23, 27, 62, 55, 31, 41], spatial attention [22, 56, 42, 9], and hybrid channel-spatial attention [57, 40, 38, 14, 20, 65, 39, 61].”
Transformers in Medical Imaging: A Survey (2022, methodology): “Specifically, the patch embedding layer of the Transformer has been re-designed using the shuffling layer [110] and group convolutions to significantly reduce the number of parameters.”