Automatic Labeling of Semantic Roles

By Daniel Gildea, Dan Jurafsky. 2000

1,507 citations recorded by the source corpus.

We present a system for identifying the semantic relationships, or semantic roles, filled by constituents of a sentence within a semantic frame. Given an input sentence and a target word and frame, the system labels constituents with either abstract semantic roles, such as Agent or Patient, or more domain-specific semantic roles, such as Speaker, Message, and Topic. The system is based on statistical classifiers trained on roughly 50,000 sentences that were hand-annotated with semantic roles by the FrameNet semantic labeling project. We then parsed each training sentence into a syntactic tree and extracted various lexical and syntactic features, including the phrase type of each constituent, its grammatical function, and its position in the sentence. These features were combined with knowledge of the predicate verb, noun, or adjective, as well as information such as the prior probabilities of various combinations of semantic roles. We used various lexical clustering algorithms to generalize across possible fillers of roles. Test sentences were parsed, were annotated with these features, and were then passed through the classifiers. Our system achieves 82 accuracy in identifying the semantic role of presegmented constituents. At the more difficult task of simultaneously segmenting constituents and identifying their semantic role, the system achieved 65 precision and 61 recall. Our study also allowed us to compare the usefulness of different features and feature combination methods in the semantic role labeling task. We also explore the integration of role labeling with statistical syntactic parsing and attempt to generalize to predicates unseen in the training data.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “candidates under consideration. There are instances of text ranking problems in “core” NLP tasks that at ﬁrst glance have nothing to do with text ranking. Consider the semantic role labeling problem [Gildea and Jurafsky, 2001, Palmer et al., 2010], where the system’s task is to populate “slots” in a”
Natural Language Processing (Almost) from Scratch (2011, background): “Feature categories commonly used by these system include (Gildea and Jurafsky, 2002; Pradhan et al., 2004): • the parts of speech and syntactic labels of words and nodes in the tree; • the node’s position (left or right) in relation to the verb; • the syntactic path to the verb in the parse tree; •…”
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling (2017, background): “Semantic role labeling (SRL) (Gildea and Jurafsky, 2002) can be informally described as the task of discovering who did what to whom.”
Automatically Solving Number Word Problems by Semantic Parsing and Reasoning (2015, background): “In semantic role labeling and frame-semantic parsing (Gildea & Jurafsky, 2002; Carreras & Marquez, 2004; Marquez et al., 2008; Baker et al., 2007; Das et al., 2014), predicate-argument structures are discovered from text as their shallow semantic representation.”
SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals (2009, background): “There are three main differences between our task (classification of semantic relations between nominals) and the related task of automatic labeling of semantic roles (Gildea and Jurafsky, 2002).”
A Structured Vector Space Model for Word Meaning in Context (2008, background): “Attention has mostly been limited to selectional preferences of verbs, which have been used for example for syntactic disambiguation (Hindle and Rooth, 1993), word sense disambiguation (Mc-Carthy and Carroll, 2003) and semantic role labeling (Gildea and Jurafsky, 2002).”
The Proposition Bank: An Annotated Corpus of Semantic Roles (2005, positive): “It is interesting to note that the semantic frames are a helpful way of generalizing between predicates; words in the same frame have been found frequently to share the same syntactic argument structure (Gildea and Jurafsky 2002).”
Question Answering as Global Reasoning Over Semantic Abstractions (2018, methodology): “17) can also be viewed as a predicate-argument structure. One key abstraction we use is the predicate-argument structure provided by Semantic Role Labeling (SRL). Many SRL systems have been designed (Gildea and Jurafsky 2002; Punyakanok, Roth, and Yih 2008) using linguistic resources such as FrameNe”
Solving logic puzzles: From robust processing to precise semantics (2004, background): “And so-called semantic parsing (Gildea and Jurafsky, 2002) provides as end output only a ﬂat clas-siﬁcation of semantic arguments of predicates, ignoring much of the semantic content, such as quantiﬁers.”
Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling (2005, methodology): “For example, indicators of the immediate syntactic types that form the argu-ment, ﬂags raised by punctuation tokens in or nearby the argument, or the governing category feature of Gildea and Jurafsky (2002).”
Challenges in Sentiment Analysis (2017, background): “Detecting these various roles is essentially a semantic role-labeling problem (Gildea & Jurafsky, 2002; M`arquez, Carreras, Litkowski, & Stevenson, 2008; Palmer, Gildea, & Xue, 2010), and it is possible that they can be modeled jointly to improve detection accuracy.”
Abstractive Cross-Language Summarization via Translation Model Enhanced Predicate Argument Structure Fusing (2016, background): “PAS for a sentence are usually automatically annotated by semantic role labelling [8], in which each predicate indicates an event and the arguments express the relevant information of this event (as shown in Fig.”
Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees (2006, background): “The common approach to learn the classification of predicate arguments relates to the extraction of features from syntactic parse trees of the training sentences [15].”
High-Order Low-Rank Tensors for Semantic Role Labeling (2015, background): “The initial work by Gildea and Jurafsky (2002) already identiﬁed a compact core set of features, which were widely adopted by the SRL community.”
Shallow Semantic Parsing Based on FrameNet, VerbNet and PropBank (2006, background): “Predicates belonging to the same FrameNet frame were proven [6] to have a coherent syntactic behavior that is also different from predicates pertaining to other frames.”
Deep Semantic Role Labeling with Self-Attention (2017, positive): “2 with AM-DIR, AM-LOC and AM-MNR, but to a lesser extent. This indicates that our model has some advantages on such difﬁcult adjunct distinction (Kingsbury, Palmer, and Marcus 2002). Related work SRL Gildea and Jurafsky (2002) developed the ﬁrst automatic semantic role labeling system based on Frame”
Semantic Frame Identification with Distributed Word Representations (2014, methodology): “Early work in frame-semantic analysis was pio-neered by Gildea and Jurafsky (2002).”
Shallow Semantic Parsing using Support Vector Machines (2004, methodology): “…problem and have applied various supervised machine learning techniques to it (Gildea and Jurafsky (2000, 2002); Blaheta and Charniak (2000); Gildea and Palmer (2002); Surdeanu et al. (2003); Gildea and Hockenmaier (2003); Chen and Rambow (2003); Fleischman and Hovy (2003); Hacioglu and…”
Unsupervised Graph-basedWord Sense Disambiguation Using Measures of Word Semantic Similarity (2007, neutral): “many NL algorithms [ 8 ,18,24,26] produce structured representations from text using lexical (and syntactic) knowledge which our platform provides.”
Neural Semantic Role Labeling with Dependency Path Embeddings (2016, background): “The task of semantic role labeling (SRL) was pioneered by Gildea and Jurafsky (2002).”