Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking
By Jianguo Zhang, Kazuma Hashimoto, Chien-Sheng Wu, Yao Wan, Philip S. Yu, Richard Socher, Caiming Xiong. 2019
179 citations recorded by the source corpus.
Dialog state tracking (DST) is a core component in task-oriented dialog systems. Existing approaches for DST mainly fall into one of two categories, namely, ontology-based and ontology-free methods. An ontology-based method selects a value from a candidate-value list for each target slot, while an ontology-free method extracts spans from dialog contexts. Recent work introduced a BERT-based model to strike a balance between the two methods by pre-defining categorical and non-categorical slots. However, it is not clear enough which slots are better handled by either of the two slot types, and the way to use the pre-trained model has not been well investigated. In this paper, we propose a simple yet effective dual-strategy model for DST, by adapting a single BERT-style reading comprehension model to jointly handle both the categorical and non-categorical slots. Our experiments on the MultiWOZ datasets show that our method significantly outperforms the BERT-based counterpart, finding that the key is a deep interaction between the domain-slot and context information. When evaluated on noisy (MultiWOZ 2.0) and cleaner (MultiWOZ 2.1) settings, our method performs competitively and robustly across the two different settings. Our method sets the new state of the art in the noisy setting, while performing more robustly than the best model in the cleaner setting. We also conduct a comprehensive error analysis on the dataset, including the effects of the dual strategy for each slot, to facilitate future research.
Subfields: Artificial Intelligence, Natural Language Processing, Dialogue & Conversational AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Andrea Madotto
- Pascale Fung
- Richard Socher
- Zhaojiang Lin
- Jianfeng Gao
- Caiming Xiong
- Chien-Sheng Wu
- Bing Liu
In Natural Language Processing
- Andrea Madotto
- Pascale Fung
- Richard Socher
- Chien-Sheng Wu
- Zhaojiang Lin
- Bing Liu
- Bryan McCann
- Caiming Xiong
In Dialogue & Conversational AI
- Andrea Madotto
- Zhaojiang Lin
- Hao Cheng
- Mari Ostendorf
- Chia-Hsuan Lee
- Dilek Z. Hakkani-Tür
- Zhou Yu
- Shikib Mehri
Citations in the corpus
- A Simple Language Model for Task-Oriented Dialogue (2020, background): “TRADE, DSTQA, DST-Picklist, and SST use the script proposed by Wu et al. [53]2.”
- Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking (2021, unknown): “0 GLAD (Zhong et al., 2018) 35.57% ∗∗ 88.1 ± 0.4% SUMBT (Lee et al., 2019) 46.65% ∗∗ 91.0 ± 1.0% DST-picklist (Zhang et al., 2019) 53.30% – Trippy (Heck et al., 2020) 55.29 ± 0.28% 92.7 ± 0.2% SimpleTOD (Hosseini-Asletal.,2020) 55.72% – CHAN (Shan et al., 2020) 58 the maximum length is 256.”
- DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue (2020, methodology): “Large-scale pre-training on open-domain dialogue has demonstrated surprising generality, with models like DialoGPT (Zhang et al., 2019b), Meena (Adiwardana et al., 2020) and Blender (Roller et al., 2020) achieving response generation performance competitive with humans in certain settings.”
- TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking (2020, methodology): “In contrast to other hybrid approaches such as Zhang et al. (2019), our memory mechanisms create candidate lists of values on-the-fly with the dialog context as only source of information, thus avoiding the use of pre-defined picklists.”
- CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems (2021, background): “In the task constraint component, we encode the constraint of candidate values for slots (e.g. are, type, price-range, etc.) that have categorical values (Zhang et al. 2020; Rastogi et al. 2020).”
- MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020, methodology): “Existing works either perform classifications for each slot over a candidate-value list (Zhang et al., 2019a) or directly generate slot values with a generative model (Lei et al., 2018; Wu et al., 2019a; Kim et al., 2019; Le et al., 2020).”
- MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines (2020, methodology): “Many models have used the number of possible slot values as the clas-sification criterion (Zhang et al., 2019).”
- Dialogue State Tracking with a Language Model using Schema-Driven Prompting (2021, methodology): “The following baseline models are considered: TRADE (Wu et al., 2019), DS-DST (Zhang et al., 2019) and Seq2Seq-DU (Feng et al., 2020).”
- Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking (2021, background): “Following recent works (Zhang et al., 2019; Rastogi et al., 2020), slots are divided into”