Embeddings for Word Sense Disambiguation: An Evaluation Study

By Ignacio Iacobacci, Mohammad Taher Pilehvar, Roberto Navigli. 2016

274 citations recorded by the source corpus.

Recent years have seen a dramatic growth in the popularity of word embeddings mainly owing to their ability to capture semantic information from massive amounts of textual content. As a result, many tasks in Natural Language Processing have tried to take advantage of the potential of these distributional models. In this work, we study how word embeddings can be used in Word Sense Disambiguation, one of the oldest tasks in Natural Language Processing and Artificial Intelligence. We propose different methods through which word embeddings can be leveraged in a state-of-the-art supervised WSD system architecture, and perform a deep analysis of how different parameters affect performance. We show how a WSD system that makes use of word embeddings alone, if designed properly, can provide significant performance improvement over a state-ofthe-art WSD system that incorporates several standard WSD features.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Deep Contextualized Word Representations (2018, methodology): “This is competitive with a state-of-the-art WSD-specific supervised model using hand crafted features (Iacobacci et al., 2016) and a task specific biLSTM that is also trained with auxiliary coarse-grained semantic labels and POS tags (Raganato et al.”
SenseBERT: Driving Some Sense into BERT (2019, background): “Such embeddings were shown to contain some amount of word-sense information (Iacobacci et al., 2016; Yuan et al., 2016; Arora et al., 2018; Le et al., 2018).”
Word Sense Disambiguation: A Unified Evaluation Framework and Empirical Comparison (2017, methodology): “Recently, more complex features based on word embeddings trained on unlabeled corpora have also been explored (Taghipour and Ng, 2015b; Rothe and Sch¨utze, 2015; Iacobacci et al., 2016).”
SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense Disambiguation (2020, methodology): “Supervised systems Supervised systems have attained state-of-the-art results across all English datasets by exploiting either SVM models (Iacobacci, Pilehvar, and Navigli 2016), or neural architectures (Melamud, Goldberger, and Dagan 2016; Raganato, Delli Bovi, and Navigli 2017; Vial, Lecouteux,…”
Neural Sequence Learning Models for Word Sense Disambiguation (2017, methodology): “Moreover, the ﬁeld has been explored in depth from different angles by means of extensive empirical studies and evaluation frameworks (Pilehvar and Navigli, 2014; Iacobacci et al., 2016; McCarthy et al., 2016; Raganato et al., 2017).”
Semantic Structure and Interpretability of Word Embeddings (2017, background): “[11], word sense disambiguation [12], sentiment analysis [13] and cross-lingual studies [14].”
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training (2016, background): “tems (Zhong and Ng, 2010; Iacobacci et al., 2016; Melamud et al., 2016), could become an obstacle when applied to large corpora.”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, unknown): “Methods to perform WSD can be roughly divided into two classes: supervised (Zhong & Ng, 2010; Iacobacci, Pilehvar, & Navigli, 2016; Yuan, Richardson, Doherty, Evans, & Altendorf, 2016; Raganato, Delli Bovi, & Navigli, 2017b; Luo, Liu, Xia, Chang, & Sui, 2018) and knowledge-based (Lesk, 1986;…”