SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals

By Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, Stan Szpakowicz. 2009

919 citations recorded by the source corpus.

We present a brief overview of the main challenges in the extraction of semantic relations from English text, and discuss the shortcomings of previous data sets and shared tasks. This leads us to introduce a new task, which will be part of SemEval-2010: multi-way classification of mutually exclusive semantic relations between pairs of common nominals. The task is designed to compare different approaches to the problem and to provide a standard testbed for future research, which can benefit many applications in Natural Language Processing.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Knowledge Enhanced Contextual Word Representations (2019, neutral): “, 2017) and SemEval 2010 Task 8 (Hendrickx et al., 2009) datasets.”
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, methodology): “The next two, SemEval and TACRED (Hendrickx et al., 2010; Zhang et al., 2017), are relation extraction tasks with a subset of data”
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables (2020, methodology): “Dataset # Explanations # Train (Supervised) # Train (Semi-supervised) # Dev # Test SemEval [21] 203 7,016 1,210 800 2,715 TACRED [22] 139 68,006 2,751 22,531 15,509 Laptop 70 1,806 135 462 638 Restaurant 75 2,830 107 720 1,120”
What do you learn from context? Probing for sentence structure in contextualized word representations (2019, methodology): “For relation classiﬁcation, we use the SemEval 2010 Task 8 dataset (Hendrickx et al., 2009), which consists of sentences sampled from English web text, labeled with a set of 9 directional relation types.”
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, neutral): “Table 3 provides a comparison of FewRel to the previous RC datasets, including SemEval-2010 Task 8 dataset (Hendrickx et al., 2009), ACE 2003-2004 dataset (Strassel”
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work? (2020, methodology): “We use the SemEval 2010 Task 8 dataset (Hendrickx et al., 2009) for this task.”
PTR: Prompt Tuning with Rules for Text Classification (2021, background): “Although these PLMs can capture rich knowledge (Jawahar et al., 2019; Hewitt and Manning, 2019; Petroni et al., 2019; Yenicelik et al., 2020) from massive corpora, a ﬁne-tuning process with extra task-speciﬁc data is still required to transfer their knowledge for downstream tasks.”
Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification (2016, methodology): “Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 207–212, Berlin, Germany, August 7-12, 2016. c©2016 Association for Computational Linguistics”
Bidirectional Long Short-Term Memory Networks for Relation Classification (2015, methodology): “In this work, we focus on the classification of semantic relations between pairs of nominals (Hendrickx et al., 2010).”
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling (2020, methodology): “2017) and SemEval 2010 Task 8 (Hendrickx et al. 2009), a sentence only contains one entity pair to classify.”
Improving Relation Extraction by Pre-trained Language Representations (2019, methodology): “Initial work used statistical classiﬁers or kernel based methods in combination with discrete syntactic features [Zelenko et al., 2003, Mintz et al., 2009, Hendrickx et al., 2010], such as part-of-speech and named entities tags, morphological features, and WordNet hypernyms.”
Chemical-induced disease relation extraction via convolutional neural network (2017, background): “Recently, on the new benchmark dataset of SemEval2010 Task 8 (23) on relation classification task, deep neural networks (24) such as convolutional neural network (CNN) have exhibited remarkable potential (25–27) on account of such methods providing an automatic way of feature representation without ”
Learning to Annotate: Modularizing Data Augmentation for Text Classifiers with Natural Language Explanations (2019, methodology): “For RE we choose two datasets, TACRED (Zhang et al., 2017) and SemEval (Hendrickx et al., 2009) in our experiments.”