Embedding Words and Senses Together via Joint Knowledge-Enhanced Training

By Massimiliano Mancini, José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli. 2016

80 citations recorded by the source corpus.

Word embeddings are widely used in Natural Language Processing, mainly due to their success in capturing semantic information from massive corpora. However, their creation process does not allow the different meanings of a word to be automatically separated, as it conflates them into a single vector. We address this issue by proposing a new model which learns word and sense embeddings jointly. Our model exploits large corpora and knowledge from semantic networks in order to produce a unified vector space of word and sense embeddings. We evaluate the main features of our approach both qualitatively and quantitatively in a variety of tasks, highlighting the advantages of the proposed method in comparison to state-of-the-art word- and sense-based models.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations (2018, background): “SW2V 13 (Mancini et al., 2017) is an extension of Word2Vec (Mikolov et al., 2013a) for jointly learning word and sense embeddings, producing a shared vector space of words and senses as a result.”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, background): “Among these, the most popular is to take the most similar pair of senses across the two words (Resnik, 1995; Pilehvar & Navigli, 2015; Mancini et al., 2017), also known as MaxSim:”
Coming to Your Senses: on Controls and Evaluation Sets in Polysemy Research (2018, methodology): “In addition, we modify two models for sense-specific polysemy representation (Li and Jurafsky, 2015; Mancini et al., 2017) to randomly assign words to senses, and observe that the effect size remains unchanged between the original and random conditions.”
Towards a Seamless Integration of Word Senses into Downstream NLP Applications (2017, background): “…and lexical resources for generating sense representations for individual meanings of words (Chen et al., 2014; Johansson and Pina, 2015; Jauhar et al., 2015; Ia-cobacci et al., 2015; Rothe and Sch ¨ utze, 2015; Camacho-Collados et al., 2016b; Mancini et al., 2016; Pilehvar and Collier, 2016).”
How Gender and Skin Tone Modifiers Affect Emoji Semantics in Twitter (2018, methodology): “Then, we leverage the SW2V (Senses and Words to Vectors) embedding model (Mancini et al., 2017) to train a joint vector space”