Corpus-based Learning of Analogies and Semantic Relations

By Peter D. Turney, Michael L. Littman. 2005

205 citations recorded by the source corpus.

We present an algorithm for learning from unlabeled text, based on the Vector Space Model (VSM) of information retrieval, that can solve verbal analogy questions of the kind found in the SAT college entrance exam. A verbal analogy has the form A:B::C:D, meaning “A is to B as C is to D”; for example, mason:stone::carpenter:wood. SAT analogy questions provide a word pair, A:B, and the problem is to select the most analogous word pair, C:D, from a set of five choices. The VSM algorithm correctly answers 47% of a collection of 374 college-level analogy questions (random guessing would yield 20% correct; the average college-bound senior high school student answers about 57% correctly). We motivate this research by applying it to a difficult problem in natural language processing, determining semantic relations in noun-modifier pairs. The problem is to classify a noun-modifier pair, such as “laser printer”, according to the semantic relation between the noun (printer) and the modifier (laser). We use a supervised nearest-neighbour algorithm that assigns a class to a given noun-modifier pair by finding the most analogous noun-modifier pair in the training data. With 30 classes of semantic relations, on a collection of 600 labeled noun-modifier pairs, the learning algorithm attains an F value of 26.5% (random guessing: 3.3%). With 5 classes of semantic relations, the F value is 43.2% (random: 20%). The performance is state-of-the-art for both verbal analogies and noun-modifier relations.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Language Models are Few-Shot Learners (2020, neutral): “to hypocrisy”. On this task GPT-3 achieves 65.2% in the few-shot setting, 59.1% in the one-shot setting, and 53.7% in the zero-shot setting, whereas the average score among college applicants was 57% [TL05] (random guessing yields 20%). As shown in Figure3.12, the results improve with scale, with th”
From Frequency to Meaning: Vector Space Models of Semantics (2010, methodology): “This approach to measuring relational similarity was introduced by Turney et al. (2003) and examined in more detail by Turney and Littman (2005).”
Linguistic Regularities in Sparse and Explicit Word Representations (2014, methodology): “…use task-speciﬁc information, by either relying on a ( word - pair, connectives ) matrix rather than the standard ( word, context ) matrix (Turney and Littman, 2005; Turney, 2006), or by treating analogy detection as a supervised learning task (Ba-roni and Lenci, 2009; Jurgens et al.,…”
SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals (2009, background): “at the second one; other researchers (Kim and Baldwin, 2005; Nakov and Hearst, 2008; Nastase et al., 2006; Turney, 2005; Turney and Littman, 2005) have used their class scheme and data set.”
Domain and Function: A Dual-Space Model of Semantic Relations and Compositions (2012, methodology): “At first the patterns in these pair–pattern matrices were generated by hand (Turney et al., 2003; Turney & Littman, 2005), but later work (Turney, 2006b) used automatically generated patterns.”
Computational Approaches to Sentence Completion (2012, methodology): “, 2003) and extensively used by other researchers (Veale, 2004; Turney and Littman, 2005; D. et al., 2009).”
Text: now in 2D! A framework for lexical expansion with contextual similarity (2013, methodology): “This gives rise to the comparison of pairs, as e.g. conducted by Turney and Littman (2005) for extracting analogies of semantic relations in what they call relational similarity.”
Word Embeddings as Metric Recovery in Semantic Spaces (2016, methodology): “In addition, we use the more difficult SAT analogy dataset (version 3) (Turney and Littman, 2005), which contains 374 questions from actual exams and guidebooks.”
Combining Heterogeneous Models for Measuring Relational Similarity (2013, methodology): “Using the same dataset as in (Nastase and Szpakowicz, 2003), Turney and Littman (2005) created a 128-dimentional feature vector for each word pair based on statistics of their co-occurrence patterns in Web documents and applied the k -NN method ( k = 1 in their work).”
Polarity Inducing Latent Semantic Analysis (2012, background): “Variants of vector space models are further analyzed in (Turney and Littman, 2005; Turney, 2006; Turney and Pantel, 2010).”
SemEval-2012 Task 2: Measuring Degrees of Relational Similarity (2012, background): “Nastase and Szpakowicz (2003) propose a two-level hierarchy for noun-modifier relations, which has been widely used (Nakov and Hearst, 2008; Nastase et al., 2006; Turney and Littman, 2005; Turney, 2005).”
Measuring Semantic Similarity by Latent Relational Analysis (2005, methodology): “The VSM was also evaluated by its performance as a distance measure in a supervised nearest neighbour classifier for noun-modifier semantic relations [Turney and Littman, 2005].”
Distributional Semantics Beyond Words: Supervised Learning of Analogy and Paraphrase (2013, background): “Measuring relational similarity is closely connected to classifying word pairs according to their semantic relations (Turney and Littman, 2005).”
A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations (2008, methodology): “In NLP, analogical algorithms have been applied to machine translation (Lepage and Denoual, 2005), morphology (Lepage, 1998), and semantic relations (Turney and Littman, 2005).”
Similarity of Semantic Relations (2006, methodology): “The VSM was also evaluated by its performance as a distance (nearness) measure in a supervised nearest neighbour classiﬁer for noun-modiﬁer semantic relations (Turney and Littman, 2005).”
Decoding Wikipedia Categories for Knowledge Acquisition (2008, methodology): “We plan to adopt a method closer to relation extraction (Yates & Etzioni, 2007), based on joining terms9 found in the corpus (Turney & Littman, 2005).”