SemEval-2012 Task 2: Measuring Degrees of Relational Similarity

By David Jurgens, Saif M. Mohammad, Peter D. Turney, Keith J. Holyoak. 2012

222 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Efficient Estimation of Word Representations in Vector Space (2013, background): “An interesting task where word vectors have recently been shown to significantly outperform the previous state of the art is the SemEval-2012 Task 2 [11].”
Learning Gender-Neutral Word Embeddings (2018, methodology): “Among the 22 gender-deﬁnition word pairs, there are 2 word pairs that 5 The pair is sampled from the list of word pairs with “SIMILAR: Coordinates” relation annotated in (Jurgens et al., 2012 are not used as a seed word during the training.”
Multi-Dimensional Gender Bias Classification (2020, background): “For example, statistical gender biases are rampant in word embeddings (Jurgens et al., 2012; Bolukbasi et al., 2016; Caliskan et al., 2017; Garg et al., 2018; Zhao et al., 2018b; Basta et al., 2019; Chaloner and Maldonado, 2019; Du et al., 2019; Gonen and Goldberg, 2019; Kaneko and Bollegala, 2019; ”
Multi-Relational Latent Semantic Analysis (2013, methodology): “We evaluate MRLSA using the class-inclusion portion of SemEval-2012 Task 2 data (Jurgens et al., 2012).”
Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation (2017, background): “The sparse prior work in natural language annotations that uses BWS involves the creation of datasets for relational similarity (Jurgens et al., 2012), word-sense disambiguation (Jurgens, 2013), and word–sentiment intensity (Kiritchenko and Mohammad, 2016a).”
Combining Heterogeneous Models for Measuring Relational Similarity (2013, methodology): “…answering, where identifying the existence of certain relations is often the core problem, measuring relational similarity provides a more ﬂexible solu-tion rather than creating relational classiﬁers for pre-deﬁned or task-speciﬁc categories of relations (Tur-ney, 2006; Jurgens et al., 2012).”
Linguistic Regularities in Continuous Space Word Representations (2013, methodology): “In addition to syntactic analogy questions, we used the SemEval-2012 Task 2, Measuring Relation Similarity (Jurgens et al., 2012), to estimate the extent to which RNNLM word vectors contain semantic information.”
UTD: Determining Relational Similarity Using Lexical Patterns (2012, methodology): “Task 2 of Se-mEval 2012 (Jurgens et al., 2012) was designed to evaluate the effectiveness of automatic approaches for determining the similarity of a pair of concepts to a speciﬁc semantic relation.”
Gender-preserving Debiasing for Pre-trained Word Embeddings (2019, background): “…use the semantic ( sem ) and syntactic ( syn ) analogies in the Google analogy dataset (Mikolov et al., 2013b) (in total contains 19,556 questions), MSR dataset (7,999 syntactic questions) (Mikolov et al., 2013d) and SemEval dataset ( SE , 79 paradigms) (Jurgens et al., 2012) as benchmark datasets.”
Word Embeddings and Gender Stereotypes in Swedish and English (2019, background): “[22, 20]) no human evaluation has been performed on the analogies.”
ANLIzing the Adversarial Natural Language Inference Dataset (2020, methodology): “Taking inspiration from these works, as well as Cooper et al. (1996), Sammons et al. (2010), LoBue and Yates (2011), Jurgens et al. (2012), Jia and Liang (2017), White et al. (2017), Naik et al. (2018) and others, our goal here is to create a flexible and hierarchical annotation scheme specifically…”
Sentiment Analysis of Short Informal Texts (2014, methodology): “Mohammad (2012) developed a classi-ﬁer to detect emotions using tweets with emotion word hashtags (e.g., #anger , #surprise ) as labeled data.”
Question Answering Using Enhanced Lexical Semantic Models (2013, methodology): “We veriﬁed the quality of Probase Is-A relations using a recently proposed SemEval task of relational similarity (Jurgens et al., 2012) in a companion paper (Zhila et al., 2013), where a subset of the data is to measure the degree of two words having a class-inclusion relation.”
SemEval-2015 Task 10: Sentiment Analysis in Twitter (2015, methodology): “One such annotation scheme is MaxDiff (Louviere, 1991), which is widely used in market surveys (Almquist and Lee, 2009); it was also used in a previous SemEval task (Jurgens et al., 2012).”