Deep Structured Neural Network for Event Temporal Relation Extraction

By Rujun Han, I-Hung Hsu, Mu Yang, Aram Galstyan, Ralph M. Weischedel, Nanyun Peng. 2019

69 citations recorded by the source corpus.

We propose a novel deep structured learning framework for event temporal relation extraction. The model consists of 1) a recurrent neural network (RNN) to learn scoring functions for pair-wise relations, and 2) a structured support vector machine (SSVM) to make joint predictions. The neural network automatically learns representations that account for long-term contexts to provide robust features for the structured model, while the SSVM incorporates domain knowledge such as transitive closure of temporal relations as constraints to make better globally consistent decisions. By jointly training the two components, our model combines the benefits of both data-driven learning and knowledge exploitation. Experimental results on three high-quality event temporal relation datasets (TCR, MATRES, and TB-Dense) demonstrate that incorporated with pre-trained contextualized embeddings, the proposed model achieves significantly better performances than the state-of-the-art methods on all three datasets. We also provide thorough ablation studies to investigate our model.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Nanyun Peng
2. Xiang Ren
3. Rujun Han
4. Shuaicheng Zhang
5. Lifu Huang
6. Qiang Ning
7. Dan Roth
8. Yejin Choi

In Management Science and Operations Research

1. Rujun Han
2. Xiang Ren
3. Nanyun Peng

Citations in the corpus

• ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning (2020, methodology): “Previous SOTA systems on these data leveraged PTLMs and structured learning (Han et al., 2019c; Wang et al., 2020; Zhou et al., 2020c; Han et al., 2020) and have substantially improved model performances, though none of them tack-led the issue of lacking event temporal knowledge in PTLMs. TORQUE…”
• Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference (2020, unknown): “…based model, RNN-ATT (Liu et al. 2019a), which applies Recurrent Neural Network plus attention mechanism; (3) Structured Prediction method, SP-ILP (Han et al. 2019; Leeuwenberg and Moens 2017) leveraging the ILP optimization; (4) Basic version of our model, CTRL, which only fine-tunes a BERT-BASE…”
• Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction (2020, unknown): “Previous work mostly leverage hard constraints for inference (Yoshikawa et al., 2009; Ning et al., 2017; Leeuwenberg and Moens, 2017; Ning et al., 2018a; Han et al., 2019a,b), where constraints such as transitivity and event-relation consistency are assumed to be absolutely correct.”
• Extracting Temporal Event Relation with Syntactic-Guided Temporal Graph Transformer (2021, methodology): “Recently, deep neural networks (Dligach et al., 2017; Tourille et al., 2017) and large-scale pre-training language models (Han et al., 2019a, 2020; Wang et al., 2020a; Zhou et al., 2020) are further employed and show state-of-the-art performance on temporal relation extraction.”