Analysing Lexical Semantic Change with Contextualised Word Representations
By Mario Giulianelli, Marco Del Tredici, R. Fern'andez. 2020
215 citations recorded by the source corpus.
This paper presents the first unsupervised approach to lexical semantic change that makes use of contextualised word representations. We propose a novel method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics. We create a new evaluation dataset and show that the model representations and the detected semantic shifts are positively correlated with human judgements. Our extensive qualitative analysis demonstrates that our method captures a variety of synchronic and diachronic linguistic phenomena. We expect our work to inspire further research in this direction.
Subfields: Artificial Intelligence, Cultural Studies, Natural Language Processing, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- William W. Cohen
- Bhuwan Dhingra
- Jacob Eisenstein
- Julian Martin Eisenschlos
- D. Gillick
- Mohit Bansal
- Kyunghyun Cho
- Ruslan Salakhutdinov
In Cultural Studies
In Natural Language Processing
- William W. Cohen
- Bhuwan Dhingra
- Jacob Eisenstein
- Julian Martin Eisenschlos
- D. Gillick
- Mohit Bansal
- Kyunghyun Cho
- Ruslan Salakhutdinov
In Representation & Self-Supervised Learning
Citations in the corpus
- Time-Aware Language Models as Temporal Knowledge Bases (2021, methodology): “While Giulianelli et al. (2020) use contextualized embeddings from BERT to model lexical semantic changes post hoc , they do not add a time-sensitive component to the language model itself.”