Positional Artefacts Propagate Through Masked Language Model Embeddings

By Ziyang Luo, Artur Kulmizev, Xiaoxi Mao. 2020

46 citations recorded by the source corpus.

In this work, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across layers. Namely, we find cases of persistent outlier neurons within BERT and RoBERTa’s hidden state vectors that consistently bear the smallest or largest values in said vectors. In an attempt to investigate the source of this information, we introduce a neuron-level analysis method, which reveals that the outliers are closely related to information captured by positional embeddings. We also pre-train the RoBERTa-base models from scratch and find that the outliers disappear without using positional embeddings. These outliers, we find, are the major cause of anisotropy of encoders’ raw vector spaces, and clipping them leads to increased similarity across vectors. We demonstrate this in practice by showing that clipped vectors can more accurately distinguish word senses, as well as lead to better sentence embeddings when mean pooling. In three supervised tasks, we find that clipping does not affect the performance.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The case for 4-bit precision: k-bit Inference Scaling Laws (2022, background): “…Outlier features that emerge in large language models (Gao et al., 2019; Timkey & van Schijndel, 2021; Bondarenko et al., 2021; Wei et al., 2022; Luo et al., 2021; Kovaleva et al., 2021; Puccetti et al., 2022) can cause large quantization errors and severe performance degradation (Dettmers et…”
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022, neutral): “Large magnitude outlier features in language models have been studied before (Timkey and van Schijndel, 2021; Bondarenko et al., 2021; Wei et al., 2022; Luo et al., 2021).”
All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality (2021, background): “Contemporaneous work from Bi´s et al. (2021) provides a useful characterization of how degenerate representations may be 2021) make observations about the contribution of positional embeddings.”
BERT Busters: Outlier Dimensions that Disrupt Transformers (2021, result): “We would like to note that in an independent study concurrent to ours, Luo et al. (2021) report a similar effect of outlier values in the embedding space of BERT and RoBERTa.”
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models (2022, background): “Recently, some papers [26, 27] indicate that the Transformer-based models hold significantly large outliers (even close to 100) and these extreme outliers behave in structured patterns (mainly gather at a few embedding dimensions and even become larger on unique tokens).”
The Impact of Positional Encoding on Length Generalization in Transformers (2023, background): “The disadvantages of explicit PEs over NoPE in length extrapolation contribute to the growing evidence that positional encodings pose challenges for Transformers (Sinha et al., 2022; Luo et al., 2021).”
Outliers Dimensions that Disrupt Transformers Are Driven by Frequency (2022, background): “Concurrently with the demonstration of the outlier phenomenon by Kovaleva et al. (2021), Luo et al. (2021) attributed the high-magnitude weights to a different source: positional embeddings rather than LayerNorm weights.”
A Simple and Effective Pruning Approach for Large Language Models (2023, result): “Our work is also related to recent studies on the existence of large magnitude outlier features in large language models (Kovaleva et al., 2021; Bondarenko et al., 2021; Timkey & Schijndel, 2021; Luo et al., 2021; Puccetti et al., 2022; Wei et al., 2022b).”