SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations between Pairs of Nominals

By Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, Stan Szpakowicz. 2010

326 citations recorded by the source corpus.

SemEval-2 Task 8 focuses on Multi-way classification of semantic relations between pairs of nominals. The task was designed to compare different approaches to semantic relation classification and to provide a standard testbed for future research. This paper defines the task, describes the training and test data and the process of their creation, lists the participating systems (10 teams, 28 runs), and discusses their results.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, positive): “For example, doubling the TACRED training data from 5000 to 10000, increases the baseline accuracy by 1.28 (p=.0017).”
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables (2020, methodology): “For relation extraction we choose two datasets, TACRED [23] and SemEval [21] in our experiments.”
Matching the Blanks: Distributional Similarity for Relation Learning (2019, methodology): “performance on the SemEval 2010 Task 8 (Hendrickx et al., 2009), KBP-37 (Zhang and Wang, 2015), and TACRED (Zhang et al.”
Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification (2016, methodology): “Experiments are conducted on SemEval-2010 Task 8 dataset (Hendrickx et al., 2009).”
FewRel 2.0: Towards More Challenging Few-Shot Relation Classification (2019, methodology): “Besides, we use SemEval-2010 task 8 dataset (Hendrickx et al., 2009) as the validation set, since both the corpora and the schema of SemEval-2010 task 8 are in different domains from the original FewRel dataset and the newly-annotated test set.”
Enriching Pre-trained Language Model with Entity Information for Relation Classification (2019, background): “Given a sequence of text (usually a sentence) s and a pair of nominals e 1 and e 2 , the objective is to identify the relation between e 1 and e 2 (Hendrickx et al., 2010).”
LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation (2018, unknown): “The relation classification dataset of the Semantic Evaluation 2010 (SemEval10) shared task 8 (Hendrickx et al., 2009) consists of 19 relations (9 directed relations and one artificial class Other), 8,000 training and 2,717 testing sentences.”
Semi-supervised Relation Extraction via Incremental Meta Self-Training (2020, methodology): “We use two public relation extraction datasets with different characteristics in our experiments: (1) Se-mEval (Hendrickx et al., 2010) For the two datasets above, their named entities in the sentences have been recognized and marked in advance.”
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling (2020, methodology): “For sentence-level RE datasets such as TACRED (Zhang et al., 2017) and SemEval 2010 Task 8 (Hendrickx et al., 2009), a sentence only contains one entity pair to classify.”
Attention Guided Graph Convolutional Networks for Relation Extraction (2019, methodology): “For the sentence-level relation extraction task, we follow the experimental settings in (Zhang et al., 2018) to evaluate our model on the TACRED dataset (Zhang et al., 2017) and Semeval-10 Task 8 (Hendrickx et al., 2010).”
Bidirectional Long Short-Term Memory Networks for Relation Classification (2015, methodology): “In this work, we focus on the classification of semantic relations between pairs of nominals (Hendrickx et al., 2010).”
Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers (2019, methodology): “We also report results on the commonly used single-relation benchmark SemEval 2010 task 8 (Hendrickx et al., 2009), where only one relation is required to be predicted from each paragraph.”
Classifying Relations by Ranking with Convolutional Neural Networks (2015, background): “More information about this dataset can be found in (Hendrickx et al., 2010).”
Combining Neural Networks and Log-linear Models to Improve Relation Extraction (2015, methodology): “Classiﬁer F SVM (Hendrickx et al., 2010) 82.2 RNN (Socher et al., 2012) 77.6 MVRNN (Socher et al., 2012) 82.4 CNN (Zeng et al., 2014) 82.7 CR-CNN (dos Santos et al., 2015) 84.1 † FCM (Gormley et al., 2015) 83.0 HybridFCM (Gormley et al., 2015) 83.4 DepNN (Liu et al., 2015) 83.6 SDP-LSTM (Xu et al.,…”