Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling

By Luheng He, Kenton C. T. Lee, Omer Levy, Luke S. Zettlemoyer. 2018

196 citations recorded by the source corpus.

Recent BIO-tagging-based neural semantic role labeling models are very high performing, but assume gold predicates as part of the input and cannot incorporate span-level features. We propose an end-to-end approach for jointly predicting all predicates, arguments spans, and the relations between them. The model makes independent decisions about what relationship, if any, holds between every possible word-span pair, and learns contextualized span representations that provide rich, shared input features for each decision. Experiments demonstrate that this approach sets a new state of the art on PropBank SRL without gold predicates.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

SpanBERT: Improving Pre-training by Representing and Predicting Spans (2019, methodology): “…with a small probability (0.1) while we always take sequences of up to 512 tokens until it reaches a document boundary.4 We also deviate from the optimization by running for 2.4M steps and using an epsilon of 1e8 for Adam (Kingma and Ba, 2015), which converges to a better set of model parameters.”
nocaps: novel object captioning at scale (2019, methodology): “As captions are decoded left-to-right, we can only use the forward representation of ELMo as input encodings rather than the full bidirectional model as in [11, 42].”
What do you learn from context? Probing for sentence structure in contextualized word representations (2019, methodology): “…signiﬁcant improvements to the state-of-the-art on several tasks, including constituency parsing (Kitaev & Klein, 2018), semantic role labeling (He et al., 2018; Strubell et al., 2018), and coreference (Lee et al., 2018), and has out-performed competing techniques (Kiros et al., 2015; Conneau…”
Linguistically-Informed Self-Attention for Semantic Role Labeling (2018, background): “Early approaches to SRL (Pradhan et al., 2005; Surdeanu et al., 2007; Johansson and Nugues, 2008; Toutanova et al., 2008) focused on developing rich sets of linguistic features as input to a linear model, often combined with complex constrained inference e.g. with an ILP (Punyakanok et al., 2008).”
I Know What You Want: Semantic Learning for Text Comprehension (2018, methodology): “In our experiments, we basically follow the same hyper-parameters for each model as the original settings from their corresponding literatures (He et al. 2018b; Peters et al. 2018; Chen et al. 2017; Clark and Gardner 2018) except those speciﬁed (e.g. SRL embedding dimension).”
Large-Scale QA-SRL Parsing (2018, background): “Recently, span-based models have proven to be useful for question answering (Lee et al., 2016) and coreference resolution (Lee et al., 2017), and PropBank SRL (He et al., 2018).”
Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension (2019, background): “…representation, where the representation is a weighted self-aligned vector bounded by the span boundary of the answer, similar to Lee et al. (2017); He et al. (2018): η = softmax(wηh I αi:βi ) scoreai = watanh(Wa ∑βi j=αi ηj−αi+1h I j ) 4We also experimented with the span-level F1 function, but…”
A general framework for information extraction using dynamic span graphs (2019, methodology): “, 2017, 2018) and semantic role labeling (He et al., 2018), the model uses a beam pruning strategy to dynamically select high-quality spans, and constructs a graph using the selected spans as nodes.”
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders (2019, methodology): “, 2012), semantic role labeling (Sutton and McCallum, 2005; He et al., 2018), machine translation (Aharoni and Goldberg, 2017; Eriguchi et al.”
Simple BERT Models for Relation Extraction and Semantic Role Labeling (2019, unknown): “Single He et al. (2018b) 89.”
AggGen: Ordering and Aggregating while Generating (2021, methodology): “We first parse the sentences into SRL propositions using the implementation of He et al. (2018).14 We consider each predicate-argument structure as a separate fact, where the predicate stands for the event and its arguments are mapped to actors, recipients, time, place, etc. (see Figure 5).”
Coreferential Reasoning Learning for Language Representation (2020, methodology): “The representations of the start token and the end token of a word typically contain the whole word’s information (Lee et al., 2017, 2018; He et al., 2018), based on which we apply the copy-based training objective on both ends of the masked word.”
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (2018, methodology): “Our model draws from recent end-to-end span-based models for coreference resolution (Lee et al., 2017, 2018) and semantic role labeling (He et al., 2018) and extends them for the multi-task framework involving the three tasks of identiﬁcation of entity, relation and coreference.”
Syntactic Scaffolds for Semantic Structures (2018, methodology): “This, along with other recent ap- proaches (He et al., 2017, 2018b) show that syntax remains useful, even with strong neural models for SRL. Coreference.”
Semantics-aware BERT for Language Understanding (2019, background): “This result nearly reaches the SOTA in (He et al., 2018).”
A Frustratingly Easy Approach for Entity and Relation Extraction (2021, unknown): “The closest work to ours is DYGIE and DY-GIE++ (Luan et al., 2019; Wadden et al., 2019), which builds on recent span-based models for coreference resolution (Lee et al., 2017) and semantic role labeling (He et al., 2018).”
Zero-Shot Entity Linking by Reading Entity Descriptions (2019, background): “Indeed, open-corpus pre-training has been shown to benefit out-of-domain performance far more than indomain performance (He et al., 2018).”