Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures

By Wenqiang Lei, Xisen Jin, Min-Yen Kan, Zhaochun Ren, Xiangnan He, Dawei Yin. 2018

349 citations recorded by the source corpus.

Existing solutions to task-oriented dialogue systems follow pipeline designs which introduces architectural complexity and fragility. We propose a novel, holistic, extendable framework based on a single sequence-to-sequence (seq2seq) model which can be optimized with supervised or reinforcement learning. A key contribution is that we design text spans named belief spans to track dialogue believes, allowing task-oriented dialogue systems to be modeled in a seq2seq way. Based on this, we propose a simplistic Two Stage CopyNet instantiation which emonstrates good scalability: significantly reducing model complexity in terms of number of parameters and training time by a magnitude. It significantly outperforms state-of-the-art pipeline-based methods on large datasets and retains a satisfactory entity match rate on out-of-vocabulary (OOV) cases where pipeline-designed competitors totally fail.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Continual Learning in Task-Oriented Dialogue Systems (2020, methodology): “In this paper, we model task-oriented dialogue systems as a seq2seq generation task (Lei et al., 2018; Byrne et al., 2020) that generates both api-calls and responses.”
Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems (2019, background): “This is also similar in spirit to the Sequicity architecture (Lei et al., 2018) where the second recurrent model takes as input the belief state in the natural language (i.”
Global-to-local Memory Pointer Networks for Task-Oriented Dialogue (2019, background): “In order to reduce human effort and scale up between domains, end-to-end dialogue systems, which input plain text and directly output system responses, have shown promising results based on recurrent neural networks (Zhao et al., 2017; Lei et al., 2018) and memory networks (Sukhbaatar et al., 2015).”
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems (2019, methodology): “Lei et al. (2018) use a Seq2Seq model to generate belief spans and the delexicalized response at the same time.”
Domain Adaptive Dialog Generation via Meta Learning (2019, methodology): “Sequicity model introduces belief spans to store values of all the informable slots and also record requestable slot names through the history.”
Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context (2019, methodology): “We call it MD-Sequicity since it only extends the belief span to support multi-domain belief tracking comparing to the original Sequic-ity model (Lei et al. 2018).”
Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models (2019, methodology): “Sequicity (Lei et al., 2018) attributed the bad performance of Manning & Eric (2017)’s method to the omission of belief tracker.”
Modeling Multi-Action Policy for Task-Oriented Dialogues (2019, neutral): “For task completion evaluation, Entity F1 score and Success F1 score (Lei et al., 2018) are reported.”
Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking (2019, background): “Another direction for multi-domain DST is based on generative approaches (Lei et al., 2018; Wu et al., 2019; Le et al., 2020) which generate slot values without relying on fixed vocabularies and”
Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning (2021, background): “So far, the Seq2Seq models have dominated the study of TDSs (Eric et al. 2017; Lei et al. 2018), since they have the ability to learn latent representations for dialog history that are easily adapted to new domains.”
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020, methodology): “Using T5-small as the backbone barely improves the overall performance of Sequicity (Lei et al., 2018), because the copy mechanism (Gu et al.”
ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems (2020, background): “, 2019) and a wide variety of models (Wen et al., 2015; Peng et al., 2017; Lei et al., 2018; Wu et al., 2019; Gao et al., 2019).”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “Recently, we have witnessed the development of end-to-end learning frameworks in dialogue systems, which have been studying for years to automatically handle the semantic information in raw natural language (Gao et al., 2019a; Lei et al., 2018; Jin et al., 2018).”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Li et al. (2016a) introduced a reinforcement learning-based online learning framework.”
End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2 (2020, background): “On the other hand, end-to-end neural models for dialogue systems (Madotto et al., 2018; Lei et al., 2018) enjoy a straightforward training approach to generating system responses, but it is difficult for goal-oriented dialogues where the system needs to interact with external systems or to generate ”
ChartDialogs: Plotting from Natural Language Instructions (2020, methodology): “The design of TPSpecs is similar to how structured representations are used for dialog state tracking (Kan et al., 2018).”