Attentional Feature Fusion

By Yimian Dai, Fabian Gieseke, Stefan Oehmcke, Yiquan Wu, Kobus Barnard. IEEE Workshop/Winter Conference on Applications of Computer Vision, 2020

979 citations recorded by the source corpus.

Feature fusion, the combination of features from different layers or branches, is an omnipresent part of modern network architectures. It is often implemented via simple operations, such as summation or concatenation, but this might not be the best choice. In this work, we propose a uniform and general scheme, namely attentional feature fusion, which is applicable for most common scenarios, including feature fusion induced by short and long skip connections as well as within Inception layers. To better fuse features of inconsistent semantics and scales, we propose a multiscale channel attention module, which addresses issues that arise when fusing features given at different scales. We also demonstrate that the initial integration of feature maps can become a bottleneck and that this issue can be alleviated by adding another level of attention, which we refer to as iterative attentional feature fusion. With fewer layers or parameters, our models outperform state-of-the-art networks on both CIFAR-100 and ImageNet datasets, which suggests that more sophisticated attention mechanisms for feature fusion hold great potential to consistently yield better results compared to their direct counterparts. Our codes and trained models are available online1.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/33667aea67898e114ca100ee3e2b2a613b8a943a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ke Chen
2. Tao Zhou
3. Sheng Fang
4. Yusong Wu
5. Zhe Li
6. Kaiyu Li
7. Shlomo Dubnov
8. Taylor Berg-Kirkpatrick

In Computer Vision and Pattern Recognition

1. Ke Chen
2. Geng Chen
3. Tao Zhou
4. Sheng Fang
5. Yusong Wu
6. Zhe Li
7. Yi Zhang
8. Kaiyu Li

Citations in the corpus

• Context-aware Cross-level Fusion Network for Camouflaged Object Detection (2021, methodology): “To address these challenges, we propose an ACFM via introducing a Multi-Scale Channel Attention (MSCA) [Dai et al., 2020] mechanism to effectively fuse cross-level features, which can exploit multi-scale information to alleviate scale changes.”
• Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation (2022, methodology): “Where α = fAFF (X global, X a local) is a factor obtained by attention feature fusion (AFF) [21], a two-branch CNN model for learning the fusion factor of two inputs.”
• Changer: Feature Interaction is What You Need for Change Detection (2022, methodology): “Additionally, several attention-based methods make feature fusion more flexible and learnable [1,11,14,22,49].”