Length-Induced Embedding Collapse in Transformer-based Models

By Yu-Qi Zhou, Sunhao Dai, Zhanshuo Cao, Xiao Zhang, Jun Xu. 2024

Subfields: Representation & Self-Supervised Learning, Model Architectures