Learning to Select Knowledge for Response Generation in Dialog Systems

By Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, Hua Wu. 2019

210 citations recorded by the source corpus.

End-to-end neural models for intelligent dialogue systems suffer from the problem of generating uninformative responses. Various methods were proposed to generate more informative responses by leveraging external knowledge. However, few previous work has focused on selecting appropriate knowledge in the learning process. The inappropriate selection of knowledge could prohibit the model from learning to make full use of the knowledge. Motivated by this, we propose an end-to-end neural model which employs a novel knowledge selection mechanism where both prior and posterior distributions over knowledge are used to facilitate knowledge selection. Specifically, a posterior distribution over knowledge is inferred from both utterances and responses, and it ensures the appropriate selection of knowledge during the training process. Meanwhile, a prior distribution, which is inferred from utterances only, is used to approximate the posterior distribution so that appropriate knowledge can be selected even without responses during the inference process. Compared with the previous work, our model can better incorporate appropriate knowledge in response generation. Experiments on both automatic and human evaluation verify the superiority of our model over previous baselines.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Hindsight: Posterior-guided training of retrievers for improved open-ended generation (2021, methodology): “Prior work has also modeled the posterior of various probabilistic models (Lian et al., 2019; Kim et al., 2020; Zhan et al., 2021) or used reinforcement learning (Zhao et al.”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “Most these studies focus on two problems: (1) knowledge selection – selecting appropriate knowledge to be incorporated in the next response given the dialog context and previously-selected knowledge [68, 77, 110, 178], and (2) knowledge-aware generation – injecting the required knowledge into a gene”
Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation (2020, methodology): “Furthermore, since SKT and Post-KS cannot inject multiple knowledge items, for further discussion, we choose experiments with WSeq and TED.”
A Bichannel Transformer with Context Encoding for Document-Driven Conversation Generation in Social Media (2020, background): “focused on the selection of knowledge for conversational response generation [21].”
Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue (2020, methodology): “nd a model to leverage the two-step procedure of selecting knowledge from the pool and generating a response based on chosen knowledge and given context. One of the most related models to ours may be Lian et al. (2019), who also focus on the knowledge selection issue in the two-stage knowledge-groun”
A Survey of Knowledge-enhanced Text Generation (2020, methodology): “, utterance) [151, 179] [85, 144, 154] [138, 173] [96, 114, 121] Question Sentence - - - [10, 44] [7, 26, 40] - - [11, 23, 48] answering (i.”
Knowledge-Grounded Dialogue Generation with Term-level De-noising (2021, methodology): “We compare KTWM with three strong baselines: Post-KS (Lian et al., 2019) uses an elaborate knowledge selection module and injects the selected knowledge into a generative model by approximating prior-distribution (i.e., p ( k | p ) ) with posterior-distribution (i.e., p ( k | p, r ) ).”
Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues? (2020, methodology): “…Network (TMN) (Dinan et al., 2019) and Disentangled Response Decoder (DRD) (Zhao et al., 2020), we also compare the pre-trained models with Incremental Transformer with Deliberation Decoder (ITDD) (Li et al., 2019) and Posterior Knowledge Selection (PostKS) (Lian et al., 2019) on automatic metrics.”
Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features (2021, background): “Some of these works investigate retrieving appropriate evidence (Lian et al., 2019; Meng et al., 2020; Kim et al., 2020), while we assume that a piece of evidence has already been retrieved and focus instead on how to craft generations that are more faithful to it.”
Explaining Toxic Text via Knowledge Enhanced Text Generation (2022, background): “Finally, Lian et al. (2019) develop probabilistic mechanisms to select knowledge from knowledge bases for response generation.”
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020, methodology): “…from knowledge graphs (Zhou et al., 2018a; Moon et al., 2019; Tuan et al., 2019), retrieved from unstructured documents (Dinan et al., 2019; Lian et al., 2019; Zhao et al., 2020; Kim et al., 2020), or extracted from visual background (Mostafazadeh et al., 2017; Shuster et al., 2018; Huber…”
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback (2023, background): “The automated feedback elicits the “follow-up correction” abilities of models such as ChatGPT and Instruct-GPT in order to produce revised responses that rank higher according to some given utility functions (e.g., groundedness as measured by KF1).”
Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters (2021, methodology): “Motivated by this, latent variables have been introduced to minimize the gap between prior and the posterior knowl- edge selection (Zhao et al. 2019; Kim, Ahn, and Kim 2019; Lian et al. 2019; Chen et al. 2020).”
CoLV: A Collaborative Latent Variable Model for Knowledge-Grounded Dialogue Generation (2021, methodology): “Observing that the diversity of knowledge selection (given a dialogue context, several pieces of knowledge are appropriate) can be dramatically raised from prior and posterior distributions over knowledge, recent studies (Lian et al., 2019; Kim et al., 2020; Chen et al., 2020) utilize posterior mech”