Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog

By David Thulke, Nico Daheim, Christian Dugast, Hermann Ney. 2021

56 citations recorded by the source corpus.

This paper summarizes our work on the first track of the ninth Dialog System Technology Challenge (DSTC 9),"Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access". The goal of the task is to generate responses to user turns in a task-oriented dialog that require knowledge from unstructured documents. The task is divided into three subtasks: detection, selection and generation. In order to be compute efficient, we formulate the selection problem in terms of hierarchical classification steps. We achieve our best results with this model. Alternatively, we employ siamese sequence embedding models, referred to as Dense Knowledge Retrieval, to retrieve relevant documents. This method further reduces the computation time by a factor of more than 100x at the cost of degradation in R@1 of 5-6% compared to the first model. Then for either approach, we use Retrieval Augmented Generation to generate responses based on multiple selected snippets and we show how the method can be used to fine-tune trained embeddings.

Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jason Aaron Edward Weston
2. Kurt Shuster
3. Arthur Szlam
4. Mohit Bansal
5. Kyunghyun Cho
6. Jianfeng Gao
7. Yoshua Bengio
8. Yejin Choi

In Retrieval-Augmented Generation

1. Arthur Szlam
2. Jason Aaron Edward Weston
3. Kurt Shuster
4. Dani Yogatama
5. Leonard Adolphs
6. Yoshua Bengio
7. Antoine Bordes

In Dialogue & Conversational AI

1. Kurt Shuster
2. Jason Aaron Edward Weston
3. Arthur Szlam

Citations in the corpus

• Retrieval Augmentation Reduces Hallucination in Conversation (2021, neutral): “We note that very recently retrieval augmented generation has been applied to task-oriented dia- logue (Thulke et al., 2021), which is in contrast to the open-domain knowledge-grounded dialogue setting we consider here.”
• Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2021, background): “, 2020), and recent neural retrieval models (Shuster et al., 2021a; Thulke et al., 2021).”
• Relational Memory-Augmented Language Models (2022, background): “, 2019; Izacard and Grave, 2021), dialogue (Dinan et al., 2019; Fan et al., 2021; Thulke et al., 2021) and machine translation (Bapna and Firat, 2019; Khandelwal et al.”