Large-Scale QA-SRL Parsing

By Nicholas FitzGerald, Julian Michael, Luheng He, Luke S. Zettlemoyer. 2018

106 citations recorded by the source corpus.

We present a new large-scale corpus of Question-Answer driven Semantic Role Labeling (QA-SRL) annotations, and the first high-quality QA-SRL parser. Our corpus, QA-SRL Bank 2.0, consists of over 250,000 question-answer pairs for over 64,000 sentences across 3 domains and was gathered with a new crowd-sourcing scheme that we show has high precision and good recall at modest cost. We also present neural models for two QA-SRL subtasks: detecting argument spans for a predicate and generating questions to label the semantic relationship. The best models achieve question accuracy of 82.6% and span-level accuracy of 77.6% (under human evaluation) on the full pipelined QA-SRL prediction task. They can also, as we show, be used to gather additional annotations at low cost.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Learning and Evaluating General Linguistic Intelligence (2019, methodology): “We consider a question-answer driven semantic role labeling task from FitzGerald et al. (2018), where semantic role labeling is presented as a span prediction problem.”
Break It Down: A Question Understanding Benchmark (2020, background): “FitzGerald et al. (2018) facilitated non-expert annotation by introducing a formalism expressed in natural language for semantic-role-labeling.”
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021, methodology): “QA-pairs have also been used in semantic role labelling, such as QA-SRL (FitzGerald et al., 2018).”
Evaluating Factuality in Generation with Dependency-level Entailment (2020, background): “Another line of work focuses on question answering-based semantic representations (FitzGerald et al., 2018; Michael et al., 2018) or generating freeform questions to capture factuality (Wang et al., 2020; Durmus et al., 2020).”
Asking without Telling: Exploring Latent Ontologies in Contextual Representations (2020, background): “Work on ontology-free, or open , representations suggests that the linguistic structure captured by traditional ontologies may be encoded in a variety of possible ways (Banko et al., 2007; He et al., 2015; Michael et al., 2018) while being annotatable at large scale (Fitzgerald et al., 2018).”
Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling (2018, methodology): “It also reinforces the strong performance of similar span embedding methods for coreference (Lee et al., 2017), suggesting that this style of models could be used for other span-span relation tasks, such as syntactic parsing (Stern et al., 2017), relation extraction (Miwa and Bansal, 2016), and QA-S”
Question Answering is a Format; When is it Useful? (2019, background): “QASRL / QAMR The motivation for this work is explicitly to make annotation cheaper by having crowd workers label semantic dependencies in sentences using natural language instead of training them in a formalism ( He et al., 2015; FitzGerald et al., 2018; Michael et al., 2018).”
Controlled Crowdsourcing for High-Quality QA-SRL Annotation (2019, methodology): “Table 4: Automatic and manually-corrected evaluation of our gold standard and Dense (Fitzgerald et al., 2018) against the integrated expert set.”
Event Extraction as Machine Reading Comprehension (2020, methodology): “For example, in QA-SRL (FitzGerald et al., 2018), the question for a predicate publish is always “Who published something?””