A General Survey on Attention Mechanisms in Deep Learning

By Gianni Brauwers, Flavius Frasincar. IEEE Transactions on Knowledge and Data Engineering, 2022

619 citations recorded by the source corpus.

Attention is an important mechanism that can be employed for a variety of deep learning models across many different domains and tasks. This survey provides an overview of the most important attention mechanisms proposed in the literature. The various attention mechanisms are explained by means of a framework consisting of a general attention model, uniform notation, and a comprehensive taxonomy of attention mechanisms. Furthermore, the various measures for evaluating attention models are reviewed, and methods to characterize the structure of attention models based on the proposed framework are discussed. Last, future work in the field of attention models is considered.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5a71bf38cf409b55b14b2d5159c0b06bef9ad603

Citations in the corpus

• Convolutional Neural Networks: A Survey (2023, neutral): “Attention mechanisms are a powerful tool for improving the performance of CNNs [178–182].”