Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction

By Rujun Han, Yichao Zhou, Nanyun Peng. 2020

43 citations recorded by the source corpus.

Extracting event temporal relations is a critical task for information extraction and plays an important role in natural language understanding. Prior systems leverage deep learning and pre-trained language models to improve the performance of the task. However, these systems often suffer from two short-comings: 1) when performing maximum a posteriori (MAP) inference based on neural models, previous systems only used structured knowledge that are assumed to be absolutely correct, i.e., hard constraints; 2) biased predictions on dominant temporal relations when training with a limited amount of data. To address these issues, we propose a framework that enhances deep neural network with distributional constraints constructed by probabilistic domain knowledge. We solve the constrained inference problem via Lagrangian Relaxation and apply it on end-to-end event temporal relation extraction tasks. Experimental results show our framework is able to improve the baseline neural network models with strong statistical significance on two widely used datasets in news and clinical domains.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiang Ren
2. Nanyun Peng
3. Rujun Han
4. Yejin Choi
5. Kai-Wei Chang
6. Jie Yang
7. Hannaneh Hajishirzi
8. Percy Liang

In Management Science and Operations Research

1. Rujun Han
2. Xiang Ren
3. Nanyun Peng

In Signal Processing

1. Xiang Ren
2. Nanyun Peng
3. Rujun Han

Citations in the corpus

• ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning (2020, background): “Previous SOTA systems on these data leveraged PTLMs and structured learning (Han et al., 2019c; Wang et al., 2020; Zhou et al., 2020c; Han et al., 2020) and have substantially improved model performances, though none of them tack-led the issue of lacking event temporal knowledge in PTLMs. TORQUE…”
• Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference (2020, unknown): “Some recent studies (Leeuwenberg and Moens 2017; Ning, Feng, and Roth 2017; Han, Zhou, and Peng 2020) convert the task to a structured prediction problem and solve it with Maximum a posteriori Inference.”