Deep Semantic Role Labeling: What Works and What’s Next

By Luheng He, Kenton C. T. Lee, Michael Lewis, Luke S. Zettlemoyer. 2017

452 citations recorded by the source corpus.

We introduce a new deep learning model for semantic role labeling (SRL) that significantly improves the state of the art, along with detailed analyses to reveal its strengths and limitations. We use a deep highway BiLSTM architecture with constrained decoding, while observing a number of recent best practices for initialization and regularization. Our 8-layer ensemble model achieves 83.2 F1 on theCoNLL 2005 test set and 83.4 F1 on CoNLL 2012, roughly a 10% relative error reduction over the previous state of the art. Extensive empirical analysis of these gains show that (1) deep models excel at recovering long-distance dependencies but can still make surprisingly obvious errors, and (2) that there is still room for syntactic parsers to improve these results.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Dissecting Contextual Word Embeddings: Architecture and Representation (2018, methodology): “Semantic Role Labeling The SRL model uses the reimplementation of He et al. (2017) from Gardner et al. (2018).”
Deep Contextualized Word Representations (2018, methodology): “…pretrained word vectors (Turian et al., 2010; Mikolov et al., 2013; Pennington et al., 2014) are a standard component of most state-of-the-art NLP architectures, including for question answering (Liu et al., 2017), textual entailment (Chen et al., 2017) and semantic role labeling (He et al., 2017).”
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019, methodology): “We used the AllenNLP implementations of state-of-theart models for all of these representations (Gardner et al., 2017; Dozat et al., 2017; He et al., 2017; Stanovsky et al., 2018).”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “State-of-the-art approaches treat SRL as a tagging problem (He et al., 2017), make use of that specific structure to constrain decoding, and mix recurrent and self-attentive layers (Tan et al.”
AllenNLP: A Deep Semantic Natural Language Processing Platform (2018, methodology): “…state-of-the-art for a wide range of tasks such as text classiﬁ-cation (Howard and Ruder, 2018), machine translation (Vaswani et al., 2017), semantic role labeling (Zhou and Xu, 2015; He et al., 2017), coreference resolution (Lee et al., 2017a), and semantic parsing (Krishnamurthy et al., 2017).”
Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “27 (He et al., 2017) Bidirectional LSTM with highway connections 83.”
Towards Open Intent Discovery for Conversational Text (2019, background): “Moreover, adding linguistic structure to existing models has been shown to improve their performance across a wide range of NLU tasks, such as word embedding [38], machine translation [4], named entity recognition [21], and semantic role labelling [16].”
Deep RNNs Encode Soft Hierarchical Syntax (2018, methodology): “Semantic Role Labeling We use the pre-trained DeepSRL model from (He et al., 2017), which was trained on the training data from the CoNLL2012 dataset.”
Linguistically-Informed Self-Attention for Semantic Role Labeling (2018, methodology): “Following He et al. (2017), we next apply a series of corrections to model predictions in order to understand which error types the gold parse resolves: e.g. Fix Labels ﬁxes labels on spans matching gold boundaries, and Merge Spans merges adjacent predicted spans into a gold span.”
Errudite: Scalable, Reproducible, and Testable Error Analysis (2019, background): “…slices of interest (e.g., accuracy over question types in machine comprehension, per-label performance in semantic role labeling) (Liu et al., 2017; He et al., 2017), and (2) counterfactual error analysis , where one modiﬁes the input data to assess if expectations are met, such as adding…”
Neural Motifs: Scene Graph Parsing with Global Context (2017, methodology): “Alternating Highway LSTMs To mitigate vanishing gradient problems as information flows upward, we add highway connections to all LSTMs [14, 41, 58].”
Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation (2018, neutral): “The tasks include sentiment analysis (Socher et al. 2013), entailment (Bowman et al. 2015), summarization (Nallapati, Xiang, and Zhou 2016), question answering (Liu et al. 2018), and semantic role labeling (He et al. 2017).”
TWEETQA: A Social Media Focused Question Answering Dataset (2019, methodology): “…tweets that heavily rely on attached media to convey information, we utilize a state-of-the-art semantic role labeling model trained on CoNLL-2005 (He et al., 2017) to analyze the predicate-argument structure of the tweets collected from news articles and keep only the tweets with more than two…”
Large-Scale QA-SRL Parsing (2018, methodology): “Similar to He et al. (2017), we make independent predictions for each token at training time, and use Viterbi decoding to enforce hard BIO-constraints 5 at test time.”
Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling (2018, methodology): “Our model is better at accurately predicting arguments that are farther away from the predicates, even compared to an ensemble model (He et al., 2017) that has a higher overall F1.”
Finding a Balanced Degree of Automation for Summary Evaluation (2021, methodology): “We use a Semantic Role Labeling (SRL) model (Carreras and Màrquez, 2005; Palmer et al., 2010; He et al., 2017; Shi and Lin, 2019) to achieve this goal.”
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (2018, background): “Deep neural networks (DNNs) excel at learning from labeled data and have achieved state of the art in a wide array of supervised NLP tasks such as dependency parsing (Dozat and Manning, 2017), named entity recognition (Lample et al., 2016), and semantic role labeling (He et al., 2017).”
Simple Recurrent Units for Highly Parallelizable Recurrence (2017, background): “For instance, the top-performing models for semantic role labeling and translation use eight recurrent layers, requiring days to train (He et al., 2017; Wu et al., 2016b).”
Deep Reinforcement Learning: An Overview (2017, background): “, 2017), and many works that study syntax and semantics of languages, see a recent example in semantic role labeling (He et al., 2017b).”