Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth

By Yihe Dong, Jean-Baptiste Cordonnier, Andreas Loukas. 2021

569 citations recorded by the source corpus.

Attention-based architectures have become ubiquitous in machine learning, yet our understanding of the reasons for their effectiveness remains limited. This work proposes a new way to understand self-attention networks: we show that their output can be decomposed into a sum of smaller terms, each involving the operation of a sequence of attention heads across layers. Using this decomposition, we prove that self-attention possesses a strong inductive bias towards"token uniformity". Specifically, without skip connections or multi-layer perceptrons (MLPs), the output converges doubly exponentially to a rank-1 matrix. On the other hand, skip connections and MLPs stop the output from degeneration. Our experiments verify the identified convergence phenomena on different variants of standard transformer architectures.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Furu Wei
2. Jiashi Feng
3. Jianfeng Gao
4. Dacheng Tao
5. Mohit Bansal
6. Xinchao Wang
7. Quoc V. Le
8. Li Dong

In Electrical and Electronic Engineering

1. Furu Wei
2. Shuicheng Yan
3. Chinmay Hegde
4. Li Dong
5. Enhua Wu
6. Anthony Man-Cho So
7. Jocelyn Chanussot
8. X. Zhang

In Model Architectures

1. Jiashi Feng
2. Shuicheng Yan
3. Yunhe Wang
4. Xinchao Wang
5. Kai Han
6. X. Zhang
7. Pan Zhou
8. Weihao Yu

Citations in the corpus

• Knowledge Neurons in Pretrained Transformers (2021, background): “Recently, Wu et al. (2019) and Dong et al. (2021) have pointed out that the feed-forward network module also matters to Transformer.”
• ConViT: improving vision transformers with soft convolutional inductive biases (2021, neutral): “After 1000 epochs, the ConViTs https://doi.org/10.1088/1742-5468/ac9830 Figure 13.”
• A Contrastive Framework for Neural Text Generation (2022, background): “their representations reside in a narrow subset of the entire space (Ethayarajh, 2019; Dong et al., 2021; Su et al., 2021c).”
• A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, background): “Bi-attention models have been found to encounter low-rank problems [30], which may limit their expressive ability to some extent.”
• A Survey of Transformers (2021, background): “[32] observe that simply stacking selfattention modules causes a rank collapse problem, leading to token-uniformity inductive bias, and that the feed-forward layer is one of the important building blocks that mitigate this issue.”
• On The Computational Complexity of Self-Attention (2022, background): “Transformers have already been shown to exhibit the universal approximation property [23], but lose expressivity unless the self-attention mechanism is accompanied with skip connections [24].”
• Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (2022, background): “But some works challenge the necessity of MHSA [121], attributing the high performance of ViTs to the proper building blocks [34], and/or dynamic sparse weights [40,116].”
• Refiner: Refining Self-attention for Vision Transformers (2021, unknown): “This phenomenon, also known as over-smoothing, has been identified by some recent studies [66, 49, 18] to largely degrade the performance of the ViTs.”
• Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (2023, background): “Finally, Dong et al. (2021) demonstrate that the attention matrix for the decoder-only is a full rank matrix since it is a triangular matrix.”
• SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers (2021, background): “CNNs or RNNs); 2) as mentioned in [27], the skip connection plays a crucial role in transformers.”
• Vision GNN: An Image is Worth Graph of Nodes (2022, background): “Given the output features of graph convolution X ∈ RN×D, the feature diversity [7] is measured as γ(X) = ‖X − 1x ‖, where x = argmin x ‖X − 1x ‖, (9)”
• MetaFormer is Actually What You Need for Vision (2021, background): “prove that without residual connections or MLPs, the output converges doubly exponentially to a rank-1 matrix [16].”
• A Survey of Visual Transformers (2021, background): “CeiT [54] and LocalViT [55] extract the locality by directly adding a depthwise convolution in FFN.”
• Multi-view 3D Reconstruction with Transformers (2021, background): “However, this convergence may further cause the problem of “attention uniformity” in deeper layers, which makes a Transformer model loses expressive power speedily with respect to network depth [7].”
• Outliers Dimensions that Disrupt Transformers Are Driven by Frequency (2022, result): “As argued by Dong et al. (2021), attention alone would map tokens to very low dimensional spaces, and in that case the outlier phenomenon would be consistent with such a mapping.”
• Understanding Robustness of Transformers for Image Classification (2021, methodology): “We then turn our attention to the action of the ViT models themselves, analyzing the evolution of information through the cascade of Transformer layers and the redundancies in their internal representations via correlation analysis and lesion studiesÐas has been done in the past for Transformers for”