Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue
By Byeongchang Kim, Jaewoo Ahn, Gunhee Kim. 2020
181 citations recorded by the source corpus.
Knowledge-grounded dialogue is a task of generating an informative response based on both discourse context and external knowledge. As we focus on better modeling the knowledge selection in the multi-turn knowledge-grounded dialogue, we propose a sequential latent variable model as the first approach to this matter. The model named sequential knowledge transformer (SKT) can keep track of the prior and posterior distribution over knowledge; as a result, it can not only reduce the ambiguity caused from the diversity in knowledge selection of conversation but also better leverage the response information for proper choice of knowledge. Our experimental results show that the proposed model improves the knowledge selection accuracy and subsequently the performance of utterance generation. We achieve the new state-of-the-art performance on Wizard of Wikipedia (Dinan et al., 2019) as one of the most large-scale and challenging benchmarks. We further validate the effectiveness of our model over existing conversation methods in another knowledge-based dialogue Holl-E dataset (Moghe et al., 2018).
Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Mohit Bansal
- Percy Liang
- Yejin Choi
- Graham Neubig
- Luke S. Zettlemoyer
- Jason Aaron Edward Weston
- Christopher D. Manning
In Retrieval-Augmented Generation
- Matei A. Zaharia
- Omar Khattab
- Christopher Potts
- Jason Aaron Edward Weston
- Kurt Shuster
- Christopher D. Manning
- Mojtaba Komeili
- Natasa Milic-Frayling
In Natural Language Processing
- Jianfeng Gao
- Mohit Bansal
- Jason Aaron Edward Weston
- Yejin Choi
- Graham Neubig
- Pascale Fung
- Christopher D. Manning
- Percy Liang
Citations in the corpus
- Hindsight: Posterior-guided training of retrievers for improved open-ended generation (2021, methodology): “Prior work has also modeled the posterior of various probabilistic models (Lian et al., 2019; Kim et al., 2020; Zhan et al., 2021) or used reinforcement learning (Zhao et al.”
- Internet-Augmented Dialogue Generation (2021, background): “Other works (Zhao et al., 2020; Kim et al., 2020; Bruyn et al., 2020) study whether knowledge selection is possible from a (small) set of knowledge.”
- Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020, methodology): “While some recent papers on knowledge-grounded dialogues have paid attention to the problem of knowledge selection (Lian et al., 2019; Kim et al., 2020; Ren et al., 2019), the knowledge selection module is either deeply coupled with the specially configured models (Lian et al., 2019; Ren et al.,…”
- A Survey of Knowledge-enhanced Text Generation (2020, background): “However, by asking human experts to label what document should be retrieved and adding the retrieval loss (resulting in a multi-task learning setting), the generation performance can be greatly improved [24, 58], though the labelling process is an extremely time-consuming and labor-intensive task.”
- Knowledge-Grounded Dialogue Generation with Term-level De-noising (2021, methodology): “Since the baseline models (Post-KS 4 , SKT 5 and TED 6 ) incorporate knowledge selections, we conduct a comparative evaluation of KTWM by incorporating knowledge specific to each base-line method.”
- Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters (2021, methodology): “Motivated by this, latent variables have been introduced to minimize the gap between prior and the posterior knowl- edge selection (Zhao et al. 2019; Kim, Ahn, and Kim 2019; Lian et al. 2019; Chen et al. 2020).”
- PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning (2020, methodology): “We use Adam (Kingma and Ba, 2015) as the optimizer, with a learning rate scheduler including a linear warmup and an invsqrt decay (Vaswani et al., 2017).”
- Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation (2020, background): “SKT (Kim et al., 2020) takes into a account context from multi-turn dialogues (current action and 2 prior turns) and considers knowledge selection as a sequential decision process.”
- Empirical study on BlenderBot 2.0’s Errors Analysis in terms of Model, Data and User-Centric Approach (2022, background): “As a result, studies such as Dinan et al. (2018) and Kim et al. (2020) that propose a dialogue modeling approach to generate knowledge-based responses have been conducted.”
- Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features (2021, background): “Some of these works investigate retrieving appropriate evidence (Lian et al., 2019; Meng et al., 2020; Kim et al., 2020), while we assume that a piece of evidence has already been retrieved and focus instead on how to craft generations that are more faithful to it.”
- Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog (2021, methodology): “In the context of the challenge (Kim et al. 2020), it is assumed that every user ut-terance can either be handled by an already existing API-based system or that it can be answered based on a set of knowledge documents K = { k 1 , . . . , k N } .”
- CoLV: A Collaborative Latent Variable Model for Knowledge-Grounded Dialogue Generation (2021, methodology): “To exploit ex-ternal knowledge efficiently for conversations, typical approaches (Dinan et al., 2019; Kim et al., 2020; Xu et al., 2020; Sun et al., 2020; Chen et al., 2020; Meng et al., 2020; Chen et al., 2021) tend to decompose this task into two streamlined sub-tasks: knowledge selection and…”