Roget's thesaurus and semantic similarity

By Mario Jarmasz, Stanialaw Szpakowicz. Recent Advances in Natural Language Processing, 2012

321 citations recorded by the source corpus.

We have implemented a system that measures semantic similarity using a computerized 1987 Roget's Thesaurus, and evaluated it by performing a few typical tests. We compare the results of these tests with those produced by WordNet-based similarity measures. One of the benchmarks is Miller and Charles' list of 30 noun pairs to which human judges had assigned similarity measures. We correlate these measures with those computed by several NLP systems. The 30 pairs can be traced back to Rubenstein and Goodenough's 65 pairs, which we have also studied. Our Roget's-based system gets correlations of .878 for the smaller and .818 for the larger list of noun pairs; this is quite close to the .885 that Resnik obtained when he employed humans to replicate the Miller and Charles experiment. We further evaluate our measure by using Roget's and WordNet to answer 80 TOEFL, 50 ESL and 300 Reader's Digest questions: the correct synonym must be selected amongst a group of four words. Our system gets 78.75%, 82.00% and 74.33% of the questions respectively.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/a67f083830790586ed41823d45a7b330d0a2fd95

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Challenge Set for Advancing Language Modeling (2012, methodology): “Jarmasz and Szpakowicz (2003) used a sophisticated thesaurus-based method and achieved state-of-the art performance on the ESL synonyms task, which is 82%.”
An Improved Model of Semantic Similarity Based on Lexical Co-Occurrence (2005, methodology): “Jarmasz and Szpakowicz (2003) have developed a semantic similarity measure that is akin to the edge-counting WordNet models, but which instead makes use of the 1987 edition of Penguin's Roget's Thesaurus of English Words and Phrases.”
From Frequency to Meaning: Vector Space Models of Semantics (2010, methodology): “…(2003) VSM system for measuring word similarity is the British National Corpus (BNC),4 whereas the main resource used in non-VSM systems for measuring word similarity (Hirst & St-Onge, 1998; Leacock & Chodrow, 1998; Jarmasz & Szpakowicz, 2003) is a lexicon, such as WordNet5 or Roget’s Thesaurus.”
WordNet::Similarity - Measuring the Relatedness of Concepts (2004, methodology): “(Jarmasz and Szpakowicz, 2003) compares measures of similarity derived from WordNet and Roget’s Thesaurus.”
Verb similarity on the taxonomy of WordNet (2006, methodology): “Most methods that calculate semantic distance using ontology or thesaurus knowledge, such as WordNet (Miller, 1995; Fellbaum, 1998) or Roget’s thesaurus (Jarmasz and Szpakowicz, 2003) fall into this category.”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, background): “…in addition to these tasks, there exist other intrinsic evaluation procedures such as synonymy selection (Landauer & Dumais, 1997; Turney, 2001; Jarmasz & Szpakowicz, 2003; Reisinger & Mooney, 2010), outlier detection (Camacho-Collados & Navigli, 2016; Blair, Merhav, & Barry, 2016; Stanovsky &…”
A New Set of Norms for Semantic Relatedness Measures (2013, methodology): “Resnik’s (1995) subsequent replication of M&C’s study similarly yielded a correlation of r = 0.96.”
Comparison of Semantic Similarity for Different Languages Using the Google n-gram Corpus and Second-Order Co-occurrence Measures (2011, methodology): “Jarmasz and Szpakowicz, [7] used a computerized version of Roget’s Thesaurus to calculate the semantic distance between the word pairs.”
Computational Approaches to Sentence Completion (2012, methodology): “Jarmasz and Szpakowicz (2003) used a sophisticated thesaurusbased method and achieved state-of-the art performance, which is 82%.”
Evaluating WordNet-based Measures of Lexical Semantic Relatedness (2006, methodology): “Unfortunately, this set is still very small, and, as Jarmasz and Szpakowicz (2003) point out, is culturally and politically biased. And the scarcely larger set of synonymy norms for nouns created by Whitten, Suter, and Frank (1979) covers only words with quite closely related senses, and hence is no”
Wikipedia-based Semantic Interpretation for Natural Language Processing (2009, methodology): “Some previous studies (Jarmasz & Szpakowicz, 2003) suggested that the word pairs comprising this collection might be culturally biased. produce a single relatedness score.8 Spearman’s rank-order correlation coefficient was used to compare computed relatedness scores with human judgements; being…”
Semantic text similarity using corpus-based word similarity and string similarity (2008, background): “measure [Jarmasz and Szpakowicz 2003], then we need to set λ to 16, as there are eight levels in the Roget thesaurus and there can be at most 16 edges between two words.”
WikiRelate! Computing Semantic Relatedness Using Wikipedia (2006, methodology): “This is the reason why we do not perform additional experiments making use of other datasets from synonymity tests such as the 80 TOEFL (Landauer & Dumais, 1997), 50 ESL (Turney, 2001) or 300 Reader's Digest Word Power Game (Jarmasz & Szpakowicz, 2003) questions.”
From distributional to semantic similarity (2004, methodology): “The hierarchy structure in Roget’s has also been used in edge counting measures of semantic similarity (Jarmasz and Szpakowicz, 2003; McHale, 1998), and for computing lexical cohesion using lexical chains (Morris and Hirst, 1991).”