A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialogue

By Christopher D. Manning, Mihail Eric. 2017

156 citations recorded by the source corpus.

Task-oriented dialogue focuses on conversational agents that participate in dialogues with user goals on domain-specific topics. In contrast to chatbots, which simply seek to sustain open-ended meaningful discourse, existing task-oriented agents usually explicitly model user intent and belief states. This paper examines bypassing such an explicit representation by depending on a latent neural embedding of state and learning selective attention to dialogue history together with copying to incorporate relevant prior context. We complement recent work by showing the effectiveness of simple sequence-to-sequence neural architectures with a copy mechanism. Our model outperforms more complex memory-augmented models by 7% in per-response generation and is on par with the current state-of-the-art on DSTC2, a real-world task-oriented dialogue dataset.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, background): “Recently, there is another line of research in task-oriented dialogue which is to build a single end-to-end system rather than connecting several modules (e.g., Eric and Manning [37], Madotto et al. [98, 101], Wu et al. [175]).”
Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking (2021, neutral): “Baseline Methods First, we compare TPEM with three widely used TDSs: Ptr-Unk (Eric and Manning, 2017), Mem2Seq (Madotto et al.”
End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient (2017, methodology): “The algorithms in this line of work generate agent utterances token by token given the dialog context [19, 6, 20].”
Key-Value Retrieval Networks for Task-Oriented Dialogue (2017, methodology): “Unlike other dialogue agents which only rely on prior dialogue history for generation (Kan-nan et al., 2016; Eric and Manning, 2017), our architecture is able to access and use database-style information, while still retaining the text generation advantages of recent neural models.”
Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems (2018, methodology): “Finally, one can ﬁnd that Seq2Seq and Ptr-Unk models are also strong base-lines, which further conﬁrms that generative meth-ods can also achieve good performance in task-oriented dialog systems (Eric and Manning, 2017).”
Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems (2018, background): “Rather then encoding the real KB entity values as in (Bordes and Weston, 2017; Eric and Manning, 2017), we only encode a summary of the query results (i.”
Global-to-local Memory Pointer Networks for Task-Oriented Dialogue (2019, methodology): “In the human-human dialogue dataset (Eric et al., 2017), GLMP is able to surpass the previous state of the art on both automatic and human evaluation, which further conﬁrms the effectiveness of our double pointers usage.”
Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning (2017, neutral): “, 2016), and copyaugmented networks (Eric and Manning, 2017).”
Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability (2017, background): “, 2017), as well as to task oriented systems (Bordes and Weston, 2016; Yang et al., 2016; Eric and Manning, 2017).”
Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog (2020, methodology): “…recent years, end-to-end methods in the literature usually take the sequence-to-sequence (Seq2Seq) model to generate a response from a dialogue history (Eric and Manning, 2017; Eric et al., 2017; Madotto et al., 2018; Wen et al., 2018; Gangi Reddy et al., 2019; Qin et al., 2019b; Wu et al., 2019a).”
Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems (2019, methodology): “Similar to sequence generative models (Sutskever et al., 2014; Manning and Eric, 2017), we use a Linear transformation layer with softmax function on the decoder output to predict probabilities”
BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling (2021, background): “[9] Mihail Eric and Christopher D Manning.”
An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking (2018, background): “Other classify-and-copy mechanisms have also been explored in (Gu et al., 2016; Gulcehre et al., 2016; Eric and Manning, 2017a), and demonstrated improved performance on various seq2seq tasks such as summarization and E2E dialogue generation.”
Personalization in Goal-Oriented Dialog (2017, background): “In addition to the baselines provided, several new techniques proposed to solve these tasks are of interest to us (Perez and Liu, 2017; Seo et al., 2017; Eric and Manning, 2017).”
Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models (2019, background): “Also, (Eric and Manning, 2017) pointed out that user utterances already contains the belief state information to generate system response, meaning it is not necessary to explicitly input belief state into the model.”
Mimic and Rephrase: Reflective Listening in Open-Ended Dialogue (2019, background): “Such models have also been shown to be effective at semantic parsing (Jia and Liang, 2016), summarization (See et al., 2017; Cao et al., 2018), and task oriented dialog (Eric and Manning, 2017).”
Large-Scale Answerer in Questioner's Mind for Visual Dialog Question Generation (2019, methodology): “In SL approaches, given the dialog histories so far, the model predicts the distribution of the responses from the task-oriented system (Eric & Manning, 2017; de Vries et al., 2017; Zhao et al., 2018).”