Zero-Shot Relation Extraction via Reading Comprehension

By Omer Levy, Minjoon Seo, Eunsol Choi, Luke S. Zettlemoyer. 2017

854 citations recorded by the source corpus.

We show that relation extraction can be reduced to answering simple reading comprehension questions, by associating one or more natural-language questions with each relation slot. This reduction has several advantages: we can (1) learn relation-extraction models by extending recent neural reading-comprehension techniques, (2) build very large training sets for those models by combining relation-specific crowd-sourced questions with distant supervision, and even (3) do zero-shot learning by extracting new relation types that are only specified at test-time, for which we have no labeled training examples. Experiments on a Wikipedia slot-filling task demonstrate that the approach can generalize to new questions for known relation types with high accuracy, and that zero-shot generalization to unseen relation types is possible, at lower accuracy levels, setting the bar for future work on this task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

U-Net: Machine Reading Comprehension with Unanswerable Questions (2018, positive): “One approach is to directly extend previous MRC models by introducing a no-answer score to the score vector of the answer span (Levy et al. 2017; Clark and Gardner 2017).”
Few-shot Learning with Retrieval Augmented Language Models (2022, methodology): “…answering: NaturalQuestions (Kwiatkowski et al., 2019), TriviaQA (Joshi et al., 2017) and HotpotQA (Yang et al., 2018); slot ﬁlling: Zero Shot RE (Levy et al., 2017) and T-REx (Elsahar et al., 2018); entity linking: AIDA CoNLL-YAGO (Hoﬀart et al., 2011); dialogue: Wizard of Wikipedia (Dinan et…”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “As with SRL, we use a dataset that maps relations to a set of questions so that relation extraction can be treated as question answering: QA-ZRE (Levy et al., 2017).”
Know What You Don’t Know: Unanswerable Questions for SQuAD (2018, methodology): “The Zero-shot Relation Extraction dataset (Levy et al., 2017) contains negative examples generated with distant supervision.”
Unifying Question Answering and Text Classification via Span Extraction (2019, methodology): “…the Stanford Question Answering Dataset (SQuAD) (Rajpurkar et al., 2016), QA Zero-shot Relationship Extraction (ZRE; we use the 0 th split) (Levy et al., 2017), QA Semantic Role Labeling (SRL) (He et al., 2015), Commonsense Question Answering (CQA) (Tal-mor et al., 2018) and the two…”
Finetuned Language Models Are Zero-Shot Learners (2021, methodology): “, 2015), relation extraction (Levy et al., 2017), coreference resolution (Wu et al.”
How Context Affects Language Models' Factual Predictions (2020, methodology): “In order to apply DrQA to the LAMA probe, we take inspiration from [Levy et al., 2017] and map each cloze-style template to a natural question template ( e.g. , “ X was born in [Mask] ” to “Where was X born?”).”
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, methodology): “Following Levy et al. (2017), we introduce a trainable no-answer bias when computing softmax.”
Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling (2022, background): “…filling, and dialogue categories: HotpotQA (Yang et al., 2018), TriviaQA (Joshi et al., 2017), Natural Questions (NQ) (Kwiatkowski et al., 2019), T-REx (Elsahar et al., 2018), Zero Shot RE (zsRE) (Levy et al., 2017)), FEVER (Thorne et al., 2018), and Wizard of Wikipedia (WoW) (Dinan et al., 2018).”
Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “Surfacing knowledge from large corpora is a crucial step when dealing with knowledge intensive language tasks (Levy et al., 2017; Dinan et al., 2019; Elsahar et al., 2018; Petroni et al., 2021), such as open-domain question answering (Voorhees et al., 1999; Joshi et al., 2017; Yang et al., 2018;…”
The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpus (2021, background): “…be considered to tackle KI tasks, such as large collections of question-answer pairs (Lewis et al., 2021; Huber et al., 2021), structured knowledge sources (Berant et al., 2013; Levy et al., 2017; Elsahar et al., 2018) or domain speciﬁc collections (Tsatsaronis et al., 2015; Saikh et al., 2021).”
Hindsight: Posterior-guided training of retrievers for improved open-ended generation (2021, background): “…problems, where the query itself has high overlap with the relevant passage and there is a unique answer, for example factoid Open-domain QA such as Natural Questions (Kwiatkowski et al., 2019), HotPotQA (Yang et al., 2018), fact-checking (Thorne et al., 2018), and slot-filling (Levy et al., 2017).”
The Power of Scale for Parameter-Efficient Prompt Tuning (2021, background): “For this task, several different question answering datasets (Dua et al., 2019; Saha et al., 2018; Lai et al., 2017; Levy et al., 2017; Kembhavi et al., 2017; Rajpurkar et al., 2016) are converted into a singular, extractive format.”
Learning Dense Representations of Phrases at Scale (2020, methodology): “We focus on using two slot ﬁlling datasets from the KILT benchmark (Petroni et al., 2021): T-REx (Elsahar et al., 2018) and zero-shot relation extraction (Levy et al., 2017).”
Autoregressive Entity Retrieval (2020, background): “…2019), HotpotQA (Yang et al., 2018c), TriviaQA (Joshi et al., 2017), ELI5 (Fan et al., 2019); slot ﬁlling with T-REx (Elsahar et al., 2018), Zero Shot RE (Levy et al., 2017); entity disambiguation on AIDA CoNLL-YAGO, WNED-WIKI and WNED-CWEB; dialogue with Wizard of Wikipedia (Dinan et al., 2019).”
KILT: a Benchmark for Knowledge Intensive Language Tasks (2020, background): “Zero Shot RE (Levy et al., 2017) is a dataset designed to translate relation extraction into a reading comprehension problem.”
Locating and Editing Factual Associations in GPT (2022, methodology): “The zsRE question answering task (Levy et al., 2017) was first used for factual knowledge evaluation by De Cao et al.”