Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs

By Angela Fan, Claire Gardent, Chloé Braud, Antoine Bordes. 2019

109 citations recorded by the source corpus.

Query-based open-domain NLP tasks require information synthesis from long and diverse web results. Current approaches extractively select portions of web text as input to Sequence-to-Sequence models using methods such as TF-IDF ranking. We propose constructing a local graph structured knowledge base for each query, which compresses the web search information and reduces redundancy. We show that by linearizing the graph into a structured input sequence, models can encode the graph representations within a standard Sequence-to-Sequence setting. For two generative tasks with very long text input, long-form question answering and multi-document summarization, feeding graph representations as input can achieve better performance than using retrieved text portions.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, methodology): “[46] construct a local knowledge graph for each question to compress the information and reduce redundancy from the retrieved documents, which can be viewed as an early trial to mitigate hallucination.”
KILT: a Benchmark for Knowledge Intensive Language Tasks (2020, background): “Likewise, while the classical approach of information extraction for populating a Knowledge Base (KB, Riedel et al., 2013; Surdeanu and Ji, 2014) seems out-of-fashion, recent results show that they remain contenders (Fan et al., 2019a; Xiong et al., 2019).”
DART: Open-Domain Structured Data Record to Text Generation (2020, background): “Such applications include explaining data records to non-experts in the healthcare domain (Cawsey et al., 1997), writing sports news (Chen and Mooney, 2008), summarizing information in multiple documents (Fan et al., 2019), and generating dialogue responses (Wen et al., 2015).”
Augmenting Transformers with KNN-Based Composite Memory for Dialog (2020, background): “Other work has investigated incorporating information retrieval in language modeling and question answering (Chen et al., 2017; Fan et al., 2019; Seo et al., 2019; Guu et al., 2020), while we focus on dialogue applications and ﬂexibly incorporating knowledge from multiple, multi-modal sources.”
FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations (2022, unknown): “Whereas different approaches encode graphs into neural models for multi-document summarization (Fan et al., 2019; Li et al., 2020; Pa-sunuru et al., 2021; Wu et al., 2021; Chen et al., 2021), AMR structures have been shown to beneﬁt both document representation and summary generation (Liu et al.,…”
A Survey of Knowledge-enhanced Text Generation (2020, background): “, utterance) [151, 179] [85, 144, 154] [138, 173] [96, 114, 121] Question Sentence - - - [10, 44] [7, 26, 40] - - [11, 23, 48] answering (i.”
Improving Text-to-Text Pre-trained Models for the Graph-to-Text Task (2020, background): “Moreover, seq2seq models for text generation tasks treat all input words in a sequential order, often directly linearizing (Konstas et al., 2017; Fan et al., 2019).”
Learning Dynamic Belief Graphs to Generalize on Text-Based Games (2020, background): “[12] propose to encode graph representations by linearizing the graph as an input sequence in NLP tasks.”