A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations

By Peter D. Turney. 2008

223 citations recorded by the source corpus.

Recognizing analogies, synonyms, antonyms, and associations appear to be four distinct tasks, requiring distinct NLP algorithms. In the past, the four tasks have been treated independently, using a wide variety of algorithms. These four semantic classes, however, are a tiny sample of the full range of semantic phenomena, and we cannot afford to create ad hoc algorithms for each semantic phenomenon; we need to seek a unified approach. We propose to subsume a broad range of phenomena under analogies. To limit the scope of this paper, we restrict our attention to the subsumption of synonyms, antonyms, and associations. We introduce a supervised corpus-based machine learning algorithm for classifying analogous word pairs, and we show that it can solve multiple-choice SAT analogy questions, TOEFL synonym questions, ESL synonym-antonym questions, and similar-associated-both questions from cognitive psychology.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Distributional Memory: A General Framework for Corpus-Based Semantics (2010, neutral): “For instance, Turney (2008) models synonymy detection with a DSM that closely resembles our W 1 W 2 × L space, whereas we tackle this task under the more standard W 1 × LW 2 view.”
Domain and Function: A Dual-Space Model of Semantic Relations and Compositions (2012, methodology): “For n-ary semantic relations (Turney, 2008a), the model should scale up with neither loss of information nor exponential growth in representation size as n, the number of terms in the relations, increases.”
Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD (2012, background): “…representations (e.g., Bullinaria & Levy, 2007; Landauer & Dumais, 1997; Levy & Bullinaria, 2001; Matveeva et al. 2005; Pado & Lapata, 2007; Rapp, 2003; Terra & Clarke, 2003; Turney, 2001, 2008), despite the level of noise in its results and atypical behavior with respect to the parameters.”
Evaluation methods for unsupervised word embeddings (2015, result): “This is different to previous work, which phrased the analogy task as a classiﬁcation problem (Turney, 2008).”
Vector Space Models of Lexical Meaning (2015, background): “There is some work on attempting to identify antonyms, and also analogies and associations, as separate from synonyms (Turney, 2008).”
Computing Word-Pair Antonymy (2008, methodology): “Turney (2008) proposed a uniform method to solve word analogy problems that require identifying synonyms, antonyms, hyper-nyms, and other lexical-semantic relations between word pairs.”
Analogy-based detection of morphological and semantic relations with word embeddings: what works and what doesn’t. (2016, methodology): “Analogies have been successfully used for detecting different semantic relations, such as synonymy and antonymy (Turney, 2008), ConceptNet relations and selectional preferences (Herdadelen and Baroni, 2009), and also for inducing morphological categories from unparsed data (Soricut and Och, 2015).”
Entailment above the word level in distributional semantics (2012, background): “Several studies have refined and extended this approach (Pantel and Ravichandran, 2004; Snow et al., 2005; Snow et al., 2006; Turney, 2008).”
Polarity Inducing Latent Semantic Analysis (2012, background): “Turney (2008) proposes a uniﬁed approach to handling analogies, synonyms, antonyms and associations by transforming the last three cases into cases of analogy.”
The (too Many) Problems of Analogical Reasoning with Word Vectors (2017, background): “…plays in human cognition for discovering new knowledge and understanding new concepts, automated analogical reasoning could become a game-changer in many ﬁelds, providing a universal mechanism for detecting linguistic relations (Turney, 2008) and word sense disambiguation (Federici et al., 1997).”
Counter-fitting Word Vectors to Linguistic Constraints (2016, unknown): “Most prior work focuses on extracting antonym pairs from text rather than exploiting them (Lin et al., 2003; Mohammad et al., 2008; Turney, 2008; Hashimoto et al., 2012; Mohammad et al., 2013).”
Sketch Algorithms for Estimating Point Queries in NLP (2012, background): “In recent years, the field of Natural Language Processing (NLP) has seen tremendous growth and interest in the use of approximation, randomization, and streaming techniques for large-scale problems (Brants et al., 2007; Turney, 2008).”
UTD: Determining Relational Similarity Using Lexical Patterns (2012, methodology): “In the vein of previous methods for determining relational similarity (Turney, 2011; Turney, 2008a; Turney, 2008b; Turney, 2005), we propose two approaches using patterns generated from the contexts in which the word pairs occur.”
Combining Heterogeneous Models for Measuring Relational Similarity (2013, background): “For instance, problems of identifying spe-ciﬁc relations between words, such as synonyms, antonyms or associations, can be reduced to measuring relational similarity compared to prototypical word pairs with the desired relation (Turney, 2008).”
Computational Approaches to Sentence Completion (2012, methodology): “Among the narrower tasks, the identification of synonyms and antonyms has been studied by (Landauer and Dumais, 1997; Mohammed et al., 2008; Mohammed et al., 2011; Turney et al., 2003; Turney, 2008), who used questions from the Test of English as a Foreign Language (TOEFL), Graduate Record Exams (GR”
Distributional Models of Word Meaning (2018, unknown): “Turney PD. 2006.”
Word Semantic Representations using Bayesian Probabilistic Tensor Factorization (2014, methodology): “Turney (2008) proposed a supervised classification method for handling analogies, then apply it to antonyms by transforming antonym pairs into analogy relations.”