A Survey of Knowledge-enhanced Text Generation
By W. Yu, Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, Meng Jiang. 2020
345 citations recorded by the source corpus.
The goal of text-to-text generation is to make machines express like a human in many applications such as conversation, summarization, and translation. It is one of the most important yet challenging tasks in natural language processing (NLP). Various neural encoder-decoder models have been proposed to achieve the goal by learning to map input text to output text. However, the input text alone often provides limited knowledge to generate the desired output, so the performance of text generation is still far from satisfaction in many real-world scenarios. To address this issue, researchers have considered incorporating (i) internal knowledge embedded in the input text and (ii) external knowledge from outside sources such as knowledge base and knowledge graph into the text generation system. This research topic is known as knowledge-enhanced text generation. In this survey, we present a comprehensive review of the research on this topic over the past five years. The main content includes two parts: (i) general methods and architectures for integrating knowledge into text generation; (ii) specific techniques and applications according to different forms of knowledge data. This survey can have broad audiences, researchers and practitioners, in academia and industry.
Subfields: Artificial Intelligence, Natural Language Processing, Natural Language Generation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Graham Neubig
- Pengfei Liu
- Mohit Bansal
- Xiang Ren
- Percy Liang
- Jianfeng Gao
- Chenguang Zhu
- Luke S. Zettlemoyer
In Natural Language Processing
- Graham Neubig
- Pengfei Liu
- Xiang Ren
- Yang Liu
- Jinlan Fu
- Chenguang Zhu
- Mohit Bansal
- Luke S. Zettlemoyer
In Natural Language Generation
Citations in the corpus
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, background): “• SupervisedAttentionKnowing to pay attention to the important information is a key step when extracting useful information from objects such as long text sequences [75, 129], images [130, 149], or knowledge bases [23, 146]).”
- Deep Bidirectional Language-Knowledge Graph Pretraining (2022, neutral): “Future work includes extending D RAGON to generation to advance KG-enhanced language generation.”
- Explaining Toxic Text via Knowledge Enhanced Text Generation (2022, background): “Building upon these, we leverage expert knowledge that comes from high-quality expert annotations of the input, and explicit knowledge from knowledge graphs and bases, as such symbolic knowledge can provide relevant information to the output text (Yu et al., 2022; Mou et al., 2016).”
- Retrieval Enhanced Model for Commonsense Generation (2021, background): “SWAG (Zellers et al., 2018) is a task of inferring the upcoming event based on a partial description using common-sense.”
- Generate rather than Retrieve: Large Language Models are Strong Context Generators (2022, background): “Recent follow-up work has mainly focused on improving the retriever (Karpukhin et al., 2020; Qu et al., 2021; Sachan et al., 2022) or the reader (Izacard & Grave, 2021; Cheng et al., 2021; Yu et al., 2022), or training the system end-to-end (Lewis et al., 2020; Singh et al., 2021).”
- Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts (2022, methodology): “Incorporating external knowledge is essential for many NLG tasks to augment the limited textual information (Yu et al., 2022c; Dong et al., 2021; Yu et al., 2022b).”
- A Unified Encoder-Decoder Framework with Entity Memory (2022, methodology): “A popular approach to incorporate knowledge into the generation process is retrieving evidence documents from external sources (Lewis et al., 2020b; Izacard and 2021; Oguz et al., 2020; Yu et al., 2022c).”
- Relational World Knowledge Representation in Contextual Language Models: A Review (2021, background): “or else consider specialized tasks like data-totext generation (Yu et al., 2020).”
- Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey (2021, background): “We refer readers to the excellent surveys [98, 209] on text generation.”
- Knowledge-Augmented Methods for Natural Language Processing (2023, methodology): “Different from NLU methods, NLG methods are typically under the encoder-decoder generation framework, which poses unique challenges for leveraging knowledge into decoding the next tokens during generation [46, 48].”
- RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generation (2025, unknown): “Correspondence to: Pengcheng Jiang < pj20@illinois.edu > , Jiawei Han < hanj@illinois.edu > .”