A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches

By Eneko Agirre, Enrique Alfonseca, Keith B. Hall, Jana Kravalova, Marius Pasca, Aitor Soroa Etxabe. 2009

985 citations recorded by the source corpus.

This paper presents and compares WordNet-based and distributional similarity approaches. The strengths and weaknesses of each approach regarding similarity and relatedness tasks are discussed, and a combination is presented. Each of our methods independently provide the best results in their class on the RG and WordSim353 datasets, and a supervised combination of them yields the best published results on all datasets. Finally, we pioneer cross-lingual similarity, showing that our methods are easily adapted for a cross-lingual task with minor losses.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Multi-Relational Latent Semantic Analysis (2013, background): “In the context of lexical semantics, VSMs provide a natural way of measuring semantic word relatedness by computing the distance between the corresponding vectors, which has been a standard approach (Agirre et al., 2009; Reisinger and Mooney, 2010; Yih and Qazvinian, 2012).”
Improving Distributional Similarity with Lessons Learned from Word Embeddings (2015, background): “, 2002) partitioned into two datasets, WordSim Similarity and WordSim Relatedness (Zesch et al., 2008; Agirre et al., 2009); Bruni et al.”
Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014, background): “Surprisingly, despite the long tradition of extensive evaluations of alternative count DSMs on standard benchmarks (Agirre et al., 2009; Baroni and Lenci, 2010; Bullinaria and Levy, 2007; Bullinaria and Levy, 2012; Sahlgren, 2006; Padó and Lapata, 2007), the existing literature contains very…”
Multimodal Distributional Semantics (2013, background): “Note that WS is also capturing a broader notion of relatedness (Agirre et al., 2009).”
Improving Vector Space Word Representations Using Multilingual Correlation (2014, background): “This data was further divided into two fragments by Agirre et al. (2009) who claimed that similarity (WS-SIM) and relatedness (WS-REL) are two different kinds of relations and should be dealt with separately.”
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020, methodology): “We use the following datasets: SimLex999 (Hill et al., 2015), Word-Sim353 (Agirre et al., 2009) which contain two evaluations, on words similarity and relatedness and ﬁnally on Mturk-771 (Halawi et al., 2012).”
Dependency-Based Word Embeddings (2014, methodology): “To that end, we use the WordSim353 dataset (Finkelstein et al., 2002; Agirre et al., 2009).”
Massively Multilingual Word Embeddings (2016, methodology): “In order to evaluate how useful the word embed-dings are for a downstream task, we use the embedding vector as a dense feature representation of each word in the input, and deliberately remove any other feature available for this word (e.g., pre-ﬁxes, sufﬁxes, part-of-speech).”
Deep Neural Networks for Learning Graph Representations (2016, methodology): “…2013) on 4 datasets, including the popular WordSim353 (Finkelstein et al. 2001), WordSim Similarity (Zesch, Müller, and Gurevych 2008) and WordSim Relatedness (Agirre et al. 2009) and MC (Miller and Charles 1991), as used in (Pennington, Socher, and Manning 2014; Levy, Goldberg, and Dagan 2015).”
Are Distributional Representations Ready for the Real World? Evaluating Word Vectors for Grounded Perceptual Meaning (2017, neutral): “The majority of the NLP work in this space has focused on the downstream task of augmenting word representations with novel grounded information, often evaluating on standard semantic similarity datasets (Agirre et al., 2009; Bruni et al., 2012; Faruqui et al., 2015; Bulat et al., 2016).”
Evaluating word embedding models: methods and experimental results (2019, neutral): “Pairs Year WS-353 [40] 353 2002 WS-353-SIM [41] 203 2009 WS-353-REL [41] 252 2009 MC-30 [42] 30 1991 RG-65 [43] 65 1965 Rare-Word (RW) [44] 2034 2013 MEN [45] 3000 2012 MTurk-287 [46] 287 2011 MTurk-771 [47] 771 2012 YP-130 [48] 130 2006 SimLex-999 [49] 999 2014 Verb-143 [50] 143 2014 SimVerb-3500 [”
Evolution of Semantic Similarity—A Survey (2020, methodology): “• WS353-Sim [1]: This dataset is a subset of WS353 containing 203 word pairs from the original 353 word pairs that are more suitable for semantic similarity algorithms specifically.”
Question Answering Using Enhanced Lexical Semantic Models (2013, methodology): “Among various word similarity models (Agirre et al., 2009; Reisinger and Mooney, 2010; Gabrilovich and Markovitch, 2007; Radinsky et al., 2011), the vector space models (VSMs) based on the idea of distributional similarity (Turney and Pantel, 2010) are often used as the core component.”
Associative Multichannel Autoencoder for Multimodal Word Representation (2018, positive): “We test the baseline and proposed models on six standard evaluation benchmarks, covering two different tasks: (i) Semantic relatedness: Men-3000 (Bruni et al., 2014) and Wordrel-252 (Agirre et al., 2009); (ii) Semantic similarity: Simlex-999 (Hill et al., 2016), Semsim-7576 (Silberer and Lap-ata, 20”
Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech (2018, positive): “We used 13 benchmarks [30] to measure word similarity, including WS-353 [31], WS-353-REL [32], WS-353-SIM , MC-30 [33], RG-65 [34], Rare-Word [35], MEN [36], MTurk-287 [37], MTurk-771 [38], YP-130 [31], SimLex-999 [39], Verb-143 [40], and SimVerb-3500 [41].”
Reducing sentiment polarity for demographic attributes in word embeddings using adversarial learning (2020, methodology): “To evaluate word vector distortion before and after debiasing, we use the WordSim353 similarity dataset developed by [1].”