BERT Busters: Outlier Dimensions that Disrupt Transformers

By Olga Kovaleva, Saurabh Kulshreshtha, Anna Rogers, Anna Rumshisky. 2021

126 citations recorded by the source corpus.

Multiple studies have shown that Transformers are remarkably robust to pruning. Contrary to this received wisdom, we demonstrate that pre-trained Transformer encoders are surprisingly fragile to the removal of a very small number of features in the layer outputs (<0.0001% of model weights). In case of BERT and other pre-trained encoder Transformers, the affected component is the scaling factors and biases in the LayerNorm. The outliers are high-magnitude normalization parameters that emerge early in pre-training and show up consistently in the same dimensional position throughout the model. We show that disabling them significantly degrades both the MLM loss and the downstream task performance. This effect is observed across several BERT-family models and other popular pre-trained Transformer architectures, including BART, XLNet and ELECTRA; we also show a similar effect in GPT-2.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

The case for 4-bit precision: k-bit Inference Scaling Laws (2022, background): “…that emerge in large language models (Gao et al., 2019; Timkey & van Schijndel, 2021; Bondarenko et al., 2021; Wei et al., 2022; Luo et al., 2021; Kovaleva et al., 2021; Puccetti et al., 2022) can cause large quantization errors and severe performance degradation (Dettmers et al., 2022a; Zeng et…”
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022, neutral): “Similarly, Kovaleva et al. (2021) attribute the appearance of outliers in BERT model family to LayerNorm, and Puccetti et al. (2022) show empirically that outlier emergence is related to the frequency of tokens in the training distribution.”
SqueezeLLM: Dense-and-Sparse Quantization (2023, background): “Among the various challenges in low-bit Transformer quantization, one key issue is the presence of outliers [29], which can unnecessarily increase the quantization range.”
A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations (2023, unknown): “For instance, some outlier features may have magnitudes up to 20 times larger than others [188], leading to incomparability issues.”
A Simple and Effective Pruning Approach for Large Language Models (2023, result): “Our work is also related to recent studies on the existence of large magnitude outlier features in large language models (Kovaleva et al., 2021; Bondarenko et al., 2021; Timkey & Schijndel, 2021; Luo et al., 2021; Puccetti et al., 2022; Wei et al., 2022b).”
Eliciting Latent Predictions from Transformers with the Tuned Lens (2023, background): “Ablating an outlier direction can drastically harm performance (Kovaleva et al., 2021), so if LogitLens relies on the presence or absence of particular outlier dimensions, the perplexity of logit lens predictions might be spuriously high.”
Outliers Dimensions that Disrupt Transformers Are Driven by Frequency (2022, background): “Kovaleva et al. (2021) also show that outliers emerge early in pre-training (after 50K steps for BERT-medium).”