Knowledge-Grounded Dialogue Generation with Pre-trained Language Models

By Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, Rui Yan. 2020

206 citations recorded by the source corpus.

We study knowledge-grounded dialogue generation with pre-trained language models. To leverage the redundant external knowledge under capacity constraint, we propose equipping response generation defined by a pre-trained language model with a knowledge selection module, and an unsupervised approach to jointly optimizing knowledge selection and response generation with unlabeled dialogues. Empirical results on two benchmarks indicate that our model can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Internet-Augmented Dialogue Generation (2021, background): “Other works (Zhao et al., 2020; Kim et al., 2020; Bruyn et al., 2020) study whether knowledge selection is possible from a (small) set of knowledge.”
A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity (2023, background): “To boost informativeness, these systems leverage external knowledge, including structured knowledge such as knowledge graphs (Zhao et al., 2020; Ji et al., 2022b) and unstructured knowledge such as free text (Xu et al., 2022, 2023).”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Zhao et al. (2020b) built a knowledge-grounded dialogue generation system based on GPT-2.”
Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generation (2021, background): “…et al., 2018; Dinan et al., 2019; Gopalakrishnan et al., 2019) addresses this issue, however, most approaches require a knowledge base (KB) to retrieve knowledge for RG (Zhou et al., 2018; Zhao et al., 2020; Eric et al., 2021), which may suffer from the limited knowledge coverage of the used KBs.”
Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog (2021, background): “Zhao et al. (2020) suggest an unsupervised approach for retrieval which does not require annotations of relevant knowledge documents.”
Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters (2021, methodology): “As it is shown in Figure 1, many recent works (Lian et al. 2019; Kim, Ahn, and Kim 2019; Roller et al. 2020; Chen et al. 2020; Zhao et al. 2020) develop the solutions comprising: (1) Knowledge Retrieval, for retrieving the related knowledge sentences from a large corpus (e.g., Wikipedia); 2)…”
Language Models are Few-shot Multilingual Learners (2021, methodology): “…have been used for various sequence generation tasks, such as summarization (Raffel et al., 2020), conversational agents (Lin et al., 2020b,a; Madotto et al., 2020a; Wu and Xiong, 2020; Hosseini-Asl et al., 2020; Lin et al., 2021b), and knowledge grounding (Chen et al., 2020; Zhao et al., 2020).”
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2023, background): “…text summarization (Liu and Lapata, 2019; Yang et al., 2020; Lewis et al., 2020; Zhang et al., 2020; He et al., 2022), dialogue generation (Bao et al., 2020; Zhao et al., 2020; OpenAI, 2022), question answering (Rajpurkar et al., 2016; Yang et al., 2018), machine translation (Wu et al., 2016), etc.”
Think Before You Speak: Using Self-talk to Generate Implicit Commonsense Knowledge for Response Generation (2021, background): “…explored ways to inject external knowledge to the RG models, they either perform knowledge retrieval upon a knowledge base (KB) (Zhou et al., 2018; Zhao et al., 2020) or cast knowledge as a latent factor in the generation (Tuan et al., 2020; Xu et al., 2021), posing limitation due to coverage in…”
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines (2022, background): “…et al., 2020; Xu et al., 2021a), steer the conversation towards speciﬁc keywords or topics (Tang et al., 2019; Gupta et al., 2022a), and ground responses in background knowledge such as persona (Song et al., 2019), emotions (Zhong et al., 2019), or documents (Zhao et al., 2020; Li et al., 2022).”
Knowledge-Grounded Dialogue Generation with Term-level De-noising (2021, background): “…techniques that use knowledge, in addition to the original posts, to improve the quality of the generated responses (Ghazvinine-jad et al. (2018), Moghe et al. (2018), Dinan et al. (2019), Galley et al. (2019), Lian et al. (2019), Zheng and Zhou (2019), Zhao et al. (2020a), Zhao et al. (2020b)).”