BAM: Bottleneck Attention Module

By Jongchan Park, Sanghyun Woo, Joon-Young Lee, In-So Kweon. British Machine Vision Conference, 2018

1,311 citations recorded by the source corpus.

Recent advances in deep neural networks have been developed via architecture search for stronger representational power. In this work, we focus on the effect of attention in general deep neural networks. We propose a simple and effective attention module, named Bottleneck Attention Module (BAM), that can be integrated with any feed-forward convolutional neural networks. Our module infers an attention map along two separate pathways, channel and spatial. We place our module at each bottleneck of models where the downsampling of feature maps occurs. Our module constructs a hierarchical attention at bottlenecks with a number of parameters and it is trainable in an end-to-end manner jointly with any feed-forward models. We validate our BAM through extensive experiments on CIFAR-100, ImageNet-1K, VOC 2007 and MS COCO benchmarks. Our experiments show consistent improvement in classification and detection performances with various models, demonstrating the wide applicability of BAM. The code and models will be publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Model Architectures

View Paper: https://arxiv.org/abs/1807.06514

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Model Architectures

Citations in the corpus

Attention Augmented Convolutional Networks (2019, methodology): “All attention augmented networks use κ = 2 υ =0 Table 2 benchmarks Attention Augmentation against channel and spatial attention mechanisms BAM [31], CBAM [46] and GALA [28] with channel reduction ratio σ = 16 on the ResNet50 architecture.”
Squeeze-and-Excitation Networks (2017, background): “Concurrent work provides an interesting study into the combined use of spatial and channel attention around convolutional [58] and bottleneck units [59].”
CBAM: Convolutional Block Attention Module (2018, methodology): “Concurrently, BAM [31] takes a similar approach, decomposing 3D attention map inference into channel and spatial.”
Transfer without Forgetting (2022, methodology): “The attention maps M(·) are computed through specific layers, whose architectural design follows the insights provided in [47].”
LambdaNetworks: Modeling Long-Range Interactions Without Attention (2021, background): “Spatial attention Conversely, spatial attention mechanisms, reweigh each position based on signals aggregated from all channels (Xu et al., 2015; Park et al., 2018; Woo et al., 2018).”
Selective Kernel Networks (2019, background): “Beyond channel, BAM [32] and CBAM [45] introduce spatial attention in a similar way.”
Robust Lightweight Facial Expression Recognition Network with Label Distribution Training (2021, methodology): “Allow for computational overheads, the channel-spatial modulator is designed based on BAM (Park et al. 2018).”
FcaNet: Frequency Channel Attention Networks (2020, methodology): “Inspired by these works, a series of works like BAM [28], DAN [9], CBAM [39], scSE [32], and CoordAtten-tion [19] are proposed to fuse spatial attention [45] and channel attention.”
Context-Gated Convolution (2019, methodology): “Many methods [48,49,51,39,25,8,4,3,33] have been recently proposed to introduce global context modeling modules into CNN architectures.”
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention (2024, neutral): “Existing plug-and-play attention methods can be primarily categorized into three types: channel attention [23, 27, 62, 55, 31, 41], spatial attention [22, 56, 42, 9], and hybrid channel-spatial attention [57, 40, 38, 14, 20, 65, 39, 61].”
DIANet: Dense-and-Implicit Attention Network (2019, methodology): “Generally in practice, DNNs with a larger number of parameters do not guarantee sufficient performance improvement.”
Pattern Recognition (2018, neutral): “CASIA-B includes 11 different views (0 ◦ , 18 ◦ , ..., 180 ◦ ) and 3 different walking conditions (normal walking, with a coat and with a bag) for a total of 124 people.”
CMT: Convolutional Neural Networks Meet Vision Transformers (2021, background): “To alleviate the limited receptive ﬁelds in prior research, some researches [20,21,41,45,59,62] incorporated attention mechanisms as an operator for adaptation between modalities.”