Entity, Relation, and Event Extraction with Contextualized Span Representations
By David Wadden, Ulme Wennberg, Yi Luan, Hannaneh Hajishirzi. 2019
684 citations recorded by the source corpus.
We examine the capabilities of a unified, multi-task framework for three information extraction tasks: named entity recognition, relation extraction, and event extraction. Our framework (called DyGIE++) accomplishes all tasks by enumerating, refining, and scoring text spans designed to capture local (within-sentence) and global (cross-sentence) context. Our framework achieves state-of-the-art results across all tasks, on four datasets from a variety of domains. We perform experiments comparing different techniques to construct span representations. Contextualized embeddings like BERT perform well at capturing relationships among entities in the same or adjacent sentences, while dynamic span graph updates model long-range cross-sentence relationships. For instance, propagating span representations via predicted coreference links can enable the model to disambiguate challenging entity mentions. Our code is publicly available at https://github.com/dwadden/dygiepp and can be easily adapted for new tasks or datasets.
Subfields: Artificial Intelligence, Management Science and Operations Research, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yejin Choi
- Luke S. Zettlemoyer
- Percy Liang
- Noah A. Smith
- Jianfeng Gao
- Hannaneh Hajishirzi
- Graham Neubig
In Management Science and Operations Research
In Information Extraction
Citations in the corpus
- GLM-130B: An Open Bilingual Pre-trained Model (2022, methodology): “We adopt ACE05 (Walker & Consortium, 2005) event extraction datasets following the setting in (Wadden et al., 2019).”
- Revisiting Few-sample BERT Fine-tuning (2020, background): “1 Introduction Fine-tuning self-supervised pre-trained models has significantly boosted state-of-the-art performance on Natural Language Processing (NLP) tasks [13, 28, 43, 51, 56].”
- Unified Structure Generation for Universal Information Extraction (2022, methodology): “If a head entity has no type, T h will be blank or steps and data split of previous works (Taillé et al., 2020; Yu et al., 2020; Wadden et al., 2019).”
- DeepStruct: Pretraining of Language Models for Structure Prediction (2022, methodology): “Comparison Methods We compare our method D EEP S TRUCT on the dataset to the following meth-ods: (i) J3EE (Nguyen and Nguyen, 2019): This method presents a joint model based on a recurrent neural network to first extract mention spans for triggers and arguments and then perform pairwise classification”
- Code4Struct: Code Generation for Few-Shot Structured Prediction from Natural Language (2022, neutral): “We follow Wadden et al. (2019); Lin et al. (2020) for dataset processing.”
- Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction (2021, neutral): “…Liao and Grishman, 2010; Hong et al., 2011, 2018; Huang and Riloff, 2012; Chen et al., 2015; Sha et al., 2016; Lin et al., 2018; Yang et al., 2019; Wang et al., 2019; Ma et al., 2020; Zhang et al., 2020c), 2) multi-task joint models (McClosky et al., 2011; Li et al., 2013, 2014; Yang and…”
- Span-based Joint Entity and Relation Extraction with Transformer Pre-training (2019, methodology): “4%, confirming that in-domain language model pre-training is beneficial, which is in line with findings of (Wadden et al. 2019).”
- Event Extraction by Answering (Almost) Natural Questions (2020, methodology): “DYGIE++ (Wadden et al., 2019) is a BERT-based framework that models text spans and captures within-sentence and cross-sentence context.”
- Lifelong Event Detection with Knowledge Transfer (2021, methodology): “Similar to (Wadden et al., 2019) consider x = ( w 1: L , s, e ) as a training instance consisting of the sentence w 1: L and span offsets ( s, e ) as described in Section 2.1, and y is the corresponding event type label.”
- Scarecrow: A Framework for Scrutinizing Machine Text (2021, methodology): “Model We use a standard span classification model inspired by Wadden et al. (2019).”
- Query and Extract: Refining Event Extraction as Type-oriented Binary Decoding (2021, methodology): “Traditional event extraction studies (Ji and Grishman, 2008; Liao and Grishman, 2010; McClosky et al., 2011; Li et al., 2013; Chen et al., 2015; Cao et al., 2015; Feng et al., 2016; Yang and Mitchell, 2016; Nguyen et al., 2016; Wadden et al., 2019; Lin et al., 2020; Wang et al., 2021) usually detect”
- The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models (2021, background): “, 2020), and information extraction (Wadden et al., 2019; Lin et al., 2020).”
- A Frustratingly Easy Approach for Entity and Relation Extraction (2021, methodology): “Compared to the previous best approaches using either global features (Lin et al., 2020) or complex neural models (e.g., MT-RNNs) (Wang and Lu, 2020), our approach is much simpler and achieves large improvements on all the datasets.”
- RadGraph: Extracting Clinical Entities and Relations from Radiology Reports (2021, methodology): “When developing benchmarks for our task, we use both a pipeline approach [19] and a joint extraction approach [25].”
- A Joint Neural Model for Information Extraction with Global Features (2020, methodology): “Experiments show that our framework achieves comparable or better results compared to the state-of-the-art end-to-end architecture (Wadden et al., 2019).”
- The Art of Prompting: Event Detection based on Type Specific Prompts (2022, methodology): “For supervised event detection, we use the same data split as the previous studies (Li et al., 2013; Wadden et al., 2019; Lin et al., 2020; Du and Cardie, 2020; Lin et al., 2020; Nguyen et al., 2021; Wang et al., 2020a) on all the three benchmark datasets.”