Making Sense of Word Embeddings

By Maria Pelevina, Nikolay Arefiev, Chris Biemann, Alexander Panchenko. 2016

142 citations recorded by the source corpus.

We present a simple yet effective approach for learning word sense embeddings. In contrast to existing techniques, which either directly learn sense representations from corpora or rely on sense inventories from lexical resources, our approach can induce a sense inventory from existing word embeddings via clustering of ego-networks of related words. An integrated WSD mechanism enables labeling of words in context with learned sense vectors, which gives rise to downstream applications. Experiments show that the performance of our method is comparable to state-of-the-art unsupervised WSD systems.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations (2018, methodology): “…of proposals have been put forward, mainly in two categories: multi-prototype embeddings (Reisinger and Mooney, 2010; Neelakantan et al., 2014; Pelevina et al., 2016), which usually leverage context clustering in order to learn distinct representations for individual meanings of words, and 1…”
Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings (2019, methodology): “Other approaches include the re-use of pre-trained word embeddings in order to induce new sense embed-dings (Pelevina et al., 2016; Remus and Biemann, 2018).”
WiC: 10, 000 Example Pairs for Evaluating Context-Sensitive Representations (2018, methodology): “For these three methods we follow the disambiguation strategy suggested by Pelevina et al. (2016): for each example we retrieve the closest sense embedding to the context vector, which is computed by averaging its contained words’ embeddings.”
Measuring Corporate Culture Using Machine Learning (2020, methodology): “We use an algorithm developed by Pelevina et al. (2016) to learn word senses from embedding vectors.”
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation (2017, methodology): “…participants and two state-of-the-art systems based on word sense embeddings: Ada-Gram (Bartunov et al., 2016) based on Bayesian stick-breaking process 5 and SenseGram (Pelevina et al., 2016) based on clustering of ego-network 5 https://github.com/sbos/AdaGram.jl generated using word embeddings 6 .”
Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings (2017, methodology): “use SenseGram4 (Pelevina et al., 2016), which has been shown to perform as good as stat-of-theart unsupervised WSD systems.”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, unknown): “, 2015b; Li & Jurafsky, 2015; Pilehvar, Camacho-Collados, Navigli, & Collier, 2017), document similarity (Wu & Giles, 2015), and word sense induction (Pelevina et al., 2016; Panchenko, Ruppert, Faralli, Ponzetto, & Biemann, 2017b) and disambiguation (Chen et al.”