SensEmbed: Learning Sense Embeddings for Word and Relational Similarity

By Ignacio Iacobacci, Mohammad Taher Pilehvar, Roberto Navigli. Annual Meeting of the Association for Computational Linguistics, 2015

296 citations recorded by the source corpus.

Word embeddings have recently gained considerable popularity for modeling words in different Natural Language Processing (NLP) tasks including semantic similarity measurement. However, notwithstanding their success, word embeddings are by their very nature unable to capture polysemy, as different meanings of a word are conflated into a single representation. In addition, their learning process usually relies on massive corpora only, preventing them from taking advantage of structured knowledge. We address both issues by proposing a multifaceted approach that transforms word embeddings to the sense level and leverages knowledge from a large semantic network for effective semantic similarity measurement. We evaluate our approach on word similarity and relational similarity frameworks, reporting state-of-the-art performance on multiple datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d691814e162b29725871a75e8390430dd4c6fd7a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Coming to Your Senses: on Controls and Evaluation Sets in Polysemy Research (2018, background): “This view is supported by several studies (Huang et al., 2012; Neelakantan et al., 2014; Chen et al., 2014; Li and Jurafsky, 2015; Iacobacci et al., 2015; Mancini et al., 2017), which argue that sense-specific representations lead to improved performance in word similarity tasks.”
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (2018, methodology): “Embeddings of synsets, or sets of cognitive synonyms, and related entities such as senses and lexemes have been widely studied, often due to the desire to account for polysemy (Rothe and Schütze, 2015; Iacobacci et al., 2015).”
Neural Sequence Learning Models for Word Sense Disambiguation (2017, background): “Recently, WSD has also been leveraged to build continuous vector representations for word senses (Chen et al., 2014; Iacobacci et al., 2015; Flekova and Gurevych, 2016).”
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation (2017, methodology): “More recently, several methods were proposed to learn sense embeddings on the basis of the sense inventory of a lexical resource (Chen et al., 2014; Rothe and Sch¨utze, 2015; Camacho-Collados et al., 2015; Iacobacci et al., 2015; Nieto Pi˜na and Johansson, 2016).”
Linear Algebraic Structure of Word Senses, with Applications to Polysemy (2016, unknown): “1 The following quote from the abstract of (Iacobacci et al., 2015) is representative of the general feeling about word embeddings: “ word embeddings are by their very nature unable to capture polysemy, as diﬀerent meanings of a word are conﬂated into a single representation.”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, methodology): “Among these, the most popular is to take the most similar pair of senses across the two words (Resnik, 1995; Pilehvar & Navigli, 2015; Mancini et al., 2017), also known as MaxSim : where S w i is a set including all senses of w i and (cid:126)s i represents the sense vector representation of the…”
Supersense Embeddings: A Unified Model for Supersense Interpretation, Prediction, and Utilization (2016, background): “In addition, the explicit supersense information could be further exploited, similarly to previous sense embedding works (Iacobacci et al., 2015; Rothe and Sch¨utze, 2015; Chen et al., 2014).”
H2@BUCC18: Parallel Sentence Extraction from Comparable Corpora Using Multilingual Sentence Embeddings (2018, background): “Monolingual distributed word representations have shown great potential in boosting the performance in several NLP tasks (Iacobacci et al., 2015; Guzmán et al., 2016; Santos et al., 2017).”
De-Conflated Semantic Representations (2016, methodology): “For this task, we can only compare against the publicly-available sense representations of Iacobacci et al. (2015), Rothe and Sch¨utze (2015), Pilehvar and Navigli (2015) and Chen et al. (2014) which are linked to the WordNet sense inventory.”
Towards a Seamless Integration of Word Senses into Downstream NLP Applications (2017, background): “individual meanings of words (Chen et al., 2014; Johansson and Pina, 2015; Jauhar et al., 2015; Iacobacci et al., 2015; Rothe and Schütze, 2015; Camacho-Collados et al., 2016b; Mancini et al., 2016; Pilehvar and Collier, 2016).”
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training (2016, methodology): “A recent approach overcame this limitation by applying an off-the-shelf disambiguation system on a corpus and then using Word2Vec to learn sense embeddings over the pre-disambiguated text (Iacobacci et al., 2015, SensEmbed).”
Embeddings for Word Sense Disambiguation: An Evaluation Study (2016, background): “Several research studies have investigated the representation of word senses, instead of words (Reisinger and Mooney, 2010; Huang et al., 2012; Camacho-Collados et al., 2015b; Iacobacci et al., 2015; Rothe and Sch¨utze, 2015).”