Multi-Level Memory for Task Oriented Dialogs

By Revanth Reddy Gangi Reddy, Danish Contractor, Dinesh Raghu, Sachindra Joshi. 2018

63 citations recorded by the source corpus.

Recent end-to-end task oriented dialog systems use memory architectures to incorporate external knowledge in their dialogs. Current work makes simplifying assumptions about the structure of the knowledge base, such as the use of triples to represent knowledge, and combines dialog utterances (context) as well as knowledge base (KB) results as part of the same memory. This causes an explosion in the memory size, and makes the reasoning over memory harder. In addition, such a memory design forces hierarchical properties of the data to be fit into a triple structure of memory. This requires the memory reader to infer relationships across otherwise connected attributes. In this paper we relax the strong assumptions made by existing architectures and separate memories used for modeling dialog context and KB results. Instead of using triples to store KB results, we introduce a novel multi-level memory architecture consisting of cells for each query and their corresponding results. The multi-level memory first addresses queries, followed by results and finally each key-value pair within a result. We conduct detailed experiments on three publicly available task oriented dialog data sets and we find that our method conclusively outperforms current state-of-the-art models. We report a 15-25% increase in both entity F1 and BLEU scores.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking (2021, neutral): “…end-to-end task-oriented dialogue systems (TDSs) with sequence-to-sequence (Seq2Seq) models (Eric and Manning, 2017; Madotto et al., 2018; Gangi Reddy et al., 2019; Qin et al., 2020; Mi et al., 2019; He et al., 2020; Wang et al., 2020; Qin et al., 2021), which have taken the state-of-the-art…”
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020, methodology): “…(Eric et al., 2017a; Madotto et al., 2018; Qin et al., 2019, 2020; Banerjee and Khapra, 2019; Neelakantan et al., 2019) or an intermediate API call to retrieve part of the KB ( API+KB ) (Bordes and Weston, 2017; Eric and Manning, 2017; Madotto et al., 2018; Reddy et al., 2019; Wu et al., 2019b).”
Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems (2020, unknown): “For the row-ent representation, we refer to (Gangi Reddy et al., 2019; Qin et al., 2019) to implement Trans-former+Row&Ent, where Bag-of-word embedding and entity-type embedding are used for the row-level representation and entity-level representation.”
BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling (2021, background): “Recently, there has been growing interest in developing deep learning-based end-to-end ToD systems [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] because they can handle complex dialogue patterns with minimal hand-crafted rules.”
Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training (2020, methodology): “Following in (Reddy et al., 2019), we divide the dataset into training/validation/testing sets with 406/135/135 dialogues respectively.”
Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems (2020, methodology): “, 2019); (5) MLM that employs a multi-level memory network for modeling dialog context and KB results separately (Reddy et al., 2019); (6) GLMP that employs a memory network with a global memory pointer and a local memory pointer to strengthen the copy ability (Wu et al.”
Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning (2021, methodology): “Following (Reddy et al. 2019), the CamRest dataset is divided into training/validation/testing sets with 406/135/135 dialogs, respectively.”
Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog (2020, background): “…recent years, end-to-end methods in the literature usually take the sequence-to-sequence (Seq2Seq) model to generate a response from a dialogue history (Eric and Manning, 2017; Eric et al., 2017; Madotto et al., 2018; Wen et al., 2018; Gangi Reddy et al., 2019; Qin et al., 2019b; Wu et al., 2019a).”
A Survey of Knowledge-enhanced Text Generation (2020, methodology): “emory. Memory augmented encoder-decoder framework has achieved promising progress for many NLG tasks. For example, MemNN is widely used for encoding dialogue history in task-oriented dialogue systems [20, 120, 153]. Such frameworks enable a decoder to retrieve information from a memory during genera”