A Bayesian Model of Diachronic Meaning Change

By Lea Frermann, Mirella Lapata. 2016

163 citations recorded by the source corpus.

Word meanings change over time and an automated procedure for extracting this information from text would be useful for historical exploratory studies, information retrieval or question answering. We present a dynamic Bayesian model of diachronic meaning change, which infers temporal word representations as a set of senses and their prevalence. Unlike previous work, we explicitly model language change as a smooth, gradual process. We experimentally show that this modeling decision is beneficial: our model performs competitively on meaning change detection tasks whilst inducing discernible word senses and their development over time. Application of our model to the SemEval-2015 temporal classification benchmark datasets further reveals that it performs on par with highly optimized task-specific systems.

Subfields: Artificial Intelligence, Cultural Studies, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cultural Studies

In Natural Language Processing

Citations in the corpus

Time-Aware Language Models as Temporal Knowledge Bases (2021, background): “…have largely replaced non-contextual word embed-dings (Peters et al., 2018; Devlin et al., 2019), the main application of diachronic word embeddings is to detect and model lexical semantic changes (e.g., Frermann and Lapata, 2016), rather than to improve temporal awareness on downstream tasks.”
Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change (2016, background): “, 2014; Xu and Kemp, 2015), as well as recent work on detecting the emergence of novel word senses (Lau et al., 2012; Mitra et al., 2014; Cook et al., 2014; Mitra et al., 2015; Frermann and Lapata, 2016).”
Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings (2019, methodology): “…in automatic semantic change detection (Tang, 2018; Kutuzov et al., 2018), using methods ranging from neural models to Bayesian learning (e.g., Frermann and Lapata, 2016), Temporal Random Indexing (e.g., Basile and McGillivray, 2018) and dynamic topic modelling (e.g., Blei and Lafferty, 2006).”
Analysing Lexical Semantic Change with Contextualised Word Representations (2020, methodology): “Among these is the recent SCAN model (Frermann and Lapata, 2016), which represents (1) the meaning of a word in a time interval as a multinomial distribution over word senses and (2) word senses as probability distributions over the vocabulary.”
Diachronic word embeddings and semantic shifts: a survey (2018, background): “Frermann and Lapata (2016) drew on these ideas to trace diachronic word senses development.”
Semantic Variation in Online Communities of Practice (2018, unknown): “A variety of techniques have been explored: Latent Semantic Analysis (Sagi et al., 2011; Jatowt and Duh, 2014), topic clustering (Wijaya and Yeniterzi, 2011) and dynamic topic modelling (Frermann and Lapata, 2016).”
Diachronic degradation of language models: Insights from social media (2018, background): “Natural languages are dynamic–they are constantly evolving and adapting to the needs of their users and the environment of their use (Frermann and Lapata, 2016).”