Classification of Semantic Relations by Humans and Machines

By Erwin Marsi, Emiel Krahmer. 2005

31 citations recorded by the source corpus.

This paper addresses the classification of semantic relations between pairs of sentences extracted from a Dutch parallel corpus at the word, phrase and sentence level. We first investigate the performance of human annotators on the task of manually aligning dependency analyses of the respective sentences and of assigning one of five semantic relations to the aligned phrases (equals, generalizes, specifies, restates and intersects). Results indicate that humans can perform this task well, with an F-score of .98 on alignment and an F-score of .95 on semantic relations (after correction). We then describe and evaluate a combined alignment and classification algorithm, which achieves an F-score on alignment of .85 (using EuroWordNet) and an F-score of .80 on semantic relation classification.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

A Decomposable Attention Model for Natural Language Inference (2016, methodology): “The neural counterpart to alignment, attention (Bahdanau et al., 2015), which is a key part of our approach, was originally proposed and has been predominantly used in conjunction with LSTMs (Rockt¨aschel et al., 2016; Wang and Jiang, 2015) and to a lesser extent with CNNs (Yin et al., 2016).”
Modeling Semantic Containment and Exclusion in Natural Language Inference (2008, positive): “The FraCaS test suite (Cooper et al., 1996) contains 346 NLI problems, divided into nine sections, each focused on a speciﬁc category of semantic phenomena (listed in table 3).”
The PASCAL Recognising Textual Entailment Challenge (2005, background): “For example, [2] and [7] define lexical and lexical-syntactic entailment subtasks and [11] define an entailment-alignment subtask.”
Learning to recognize features of valid textual entailments (2006, background): “Although developed independently, the same division between alignment and classification has also been proposed by Marsi and Krahmer (2005), whose textual system is developed and evaluated on parallel translations into Dutch.”
The Third PASCAL Recognizing Textual Entailment Challenge (2007, background): “For example, [2] and [7] define lexical and lexical-syntactic entailment subtasks and [ 11 ] define an entailment-alignment subtask.”
A Phrase-Based Alignment Model for Natural Language Inference (2008, background): “Increasingly, however, the most successful RTE systems have made the alignment problem explicit.”
Natural Logic for Textual Inference (2007, methodology): “Our natural logic system, dubbed the NatLog system, has a three-stage architecture similar to those in (Marsi and Krahmer, 2005; MacCartney et al., 2006), comprising (1) linguistic pre-preprocessing, (2) alignment, and (3) entailment classification.”