A Unified Model for Word Sense Representation and Disambiguation

By Xinxiong Chen, Zhiyuan Liu, Maosong Sun. 2014

335 citations recorded by the source corpus.

Most word representation methods assume that each word owns a single semantic vector. This is usually problematic because lexical ambiguity is ubiquitous, which is also the problem to be resolved by word sense disambiguation. In this paper, we present a unified model for joint word sense representation and disambiguation, which will assign distinct representations for each word sense. 1 The basic idea is that both word sense representation (WSR) and word sense disambiguation (WSD) will benefit from each other: (1) highquality WSR will capture rich information about words and senses, which should be helpful for WSD, and (2) high-quality WSD will provide reliable disambiguated corpora for learning better sense representations. Experimental results show that, our model improves the performance of contextual word similarity compared to existing WSR methods, outperforms stateof-the-art supervised methods on domainspecific WSD, and achieves competitive performance on coarse-grained all-words WSD.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

context2vec: Learning Generic Context Embedding with Bidirectional LSTM (2016, background): “…setting, such representations were found useful for measuring context-sensitive similarity (Huang et al., 2012), word sense disambiguation (Chen et al., 2014), word sense induction (K˚ageb¨ack et al., 2015), lexical substitution (Melamud et al., 2015b), sentence completion (Liu et al.,…”
Knowledge Enhanced Contextual Word Representations (2019, background): “, 2016), use entity contexts (Chen et al., 2014; Ganea and Hofmann, 2017), or a combination of contexts and the KB (Wang et al.”
Coming to Your Senses: on Controls and Evaluation Sets in Polysemy Research (2018, background): “…et al., 2012; Neelakantan et al., 2014; Li and Jurafsky, 2015), or relied on external lexical resources (e.g., WordNet or BabelNet) which guarantee that the detected senses are mapped to true sense distinctions in natural language (Chen et al., 2014; Iacobacci et al., 2015; Mancini et al., 2017).”
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation (2017, methodology): “More recently, several methods were proposed to learn sense embeddings on the basis of the sense inventory of a lexical resource (Chen et al., 2014; Rothe and Sch¨utze, 2015; Camacho-Collados et al., 2015; Iacobacci et al., 2015; Nieto Pi˜na and Johansson, 2016).”
Linked Disambiguated Distributional Semantic Networks (2016, methodology): “These include approaches that modify Word2Vec [15] to learn sense embeddings [5], methods to enrich WordNet with embeddings for synsets and lexemes [21], acquire continuous word representations by combining random walks over knowledge bases and neural language models [11], or produce joint lexical a”
Neural Sequence Learning Models for Word Sense Disambiguation (2017, methodology): “Both BLSTM and Seq2Seq outperformed UKB (Agirre et al., 2014) and IMS trained on SemCor (Taghipour and Ng, 2015a), as well as recent supervised approaches based on distributional semantics and neural architectures (Chen et al., 2014; Yuan et al., 2016).”
Supersense Embeddings: A Unified Model for Supersense Interpretation, Prediction, and Utilization (2016, background): “Disambiguated word+supersense vectors of annotated words can be employed similarly to sense embeddings (Iacobacci et al., 2015; Chen et al., 2014) to improve downstream tasks and serve as input for supersense disambiguation or contextual similarity systems”
Embeddings for Word Sense Disambiguation: An Evaluation Study (2016, methodology): “The ability of embeddings to capture knowledge has been exploited in several tasks, such as Machine Translation (Mikolov et al., 2013, MT), Sentiment Analysis (Socher et al., 2013), Word Sense Disambiguation (Chen et al., 2014, WSD) and Language Understanding (Mesnil et al., 2013).”
De-Conflated Semantic Representations (2016, methodology): “For this task, we can only compare against the publicly-available sense representations of Iacobacci et al. (2015), Rothe and Sch¨utze (2015), Pilehvar and Navigli (2015) and Chen et al. (2014) which are linked to the WordNet sense inventory.”
Word Sense Disambiguation: A Unified Evaluation Framework and Empirical Comparison (2017, background): “…same principle, various works have adapted the original algorithm by also taking into account deﬁnitions from related words (Banerjee and Pedersen, 2003), or by calculating the distributional similarity between definitions and the context of the target word (Basile et al., 2014; Chen et al., 2014).”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, background): “…Navigli, & Collier, 2017), document similarity (Wu & Giles, 2015), and word sense induction (Pelevina et al., 2016; Panchenko, Ruppert, Faralli, Ponzetto, & Biemann, 2017b) and disambiguation (Chen et al., 2014; Rothe & Sch¨utze, 2015; Camacho-Collados et al., 2016; Peters et al., 2018).”
AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes (2015, methodology): “A simple baseline is to use the underlying word2vec embeddings directly (line 5).”
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training (2016, methodology): “In order to obtain a representation for each sense of a word, a number of approaches have leveraged lexical resources to learn sense embeddings as a result of post-processing conventional word embeddings (Chen et al., 2014; Johansson and Pina, 2015; Jauhar et al., 2015; Rothe and Schütze, 2015; Pile”
Towards a Seamless Integration of Word Senses into Downstream NLP Applications (2017, background): “individual meanings of words (Chen et al., 2014; Johansson and Pina, 2015; Jauhar et al., 2015; Iacobacci et al., 2015; Rothe and Schütze, 2015; Camacho-Collados et al., 2016b; Mancini et al., 2016; Pilehvar and Collier, 2016).”
SenseBERT: Driving Some Sense into BERT (2019, methodology): “WordNet) have been employed, relying on hypernymity relations (Rothe and Schütze, 2015) or the gloss for each sense (Chen et al., 2014).”
Under review as a conference paper at ICLR 2016 (2015, background): “Other notable extensions include Kiros et al. (2014), which employed tensors to incorporate meta-data into the embedding, and Pennington et al.”
Reading Twice for Natural Language Understanding (2017, background): “Following studies [17, 5], however, quickly realized that a single representation can hardly account for lexical ambiguity and proposed to model words by multiple representations instead.”