MultiWOZ 2.1: Multi-Domain Dialogue State Corrections and State Tracking Baselines

By Mihail Eric, Rahul Goel, Shachi Paul, Adarsh Kumar, Abhishek Sethi, Anuj Kumar Goyal, Peter Ku, Sanchit Agarwal, Shuyang Gao, Dilek Z. Hakkani-Tür. 2019

192 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Semantic Noise Matters for Neural Natural Language Generation (2019, background): “These ﬁndings suggest that efforts to create more accurate training data—whether through stricter crowd-sourcing protocols, conducting follow-up annotations (cf. Eric et al., 2019), or automated cleanup heuristics like we report here—are likely necessary in the NLG and dialogue systems communities.”
A Simple Language Model for Task-Oriented Dialogue (2020, neutral): “Recently, [14] released MultiWOZ 2.”
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, background): “• Conversational tasks require understanding of not only the user’s last request but also the full interaction history between users and machines (Budzianowski et al., 2018; Eric et al., 2019; Yu et al., 2019a).”
Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset (2019, methodology): “By us-ing large pre-trained models like BERT (Devlin et al. 2019), our model can generalize to unseen services and is robust to API changes, while achieving competitive results on the original and updated MultiWOZ datasets (Eric et al. 2019).”
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020, methodology): “However, the recently proposed multi-domain task-oriented dialogue datasets (Budzianowski et al., 2018; Eric et al., 2019) bring new challenges for multi-domain dialogue state tracking and response generation.”
TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking (2020, methodology): “With our slot-value copy mechanism to resolve coreferring value phrases, we employ a method which is reminiscent of Gao et al. (2019)’s slot carryover, but with the sharp distinction that we copy values between different slots, facilitating value sharing within and across domains.”
Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering (2019, methodology): “For example, the MultiWOZ 2.0/2.1 datasets (Budzianowski et al., 2018; Eric et al., 2019) have 30 (domain, slot) pairs and more than 4, 500 values (Wu et al., 2019).”
Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking (2020, methodology): “The MultiWOZ dataset still has signiﬁcant inconsistencies (Zhou and Small, 2019) despite having been constructed through multiple rounds of annotations (Budzianowski et al., 2018; Eric et al., 2019).”
Dialogue Transformers (2019, methodology): “In the present section we establish that both approaches are valid, based on the recently released MultiWOZ 2.1 dataset [22, 23].”
Dialogue State Tracking with a Language Model using Schema-Driven Prompting (2021, background): “Since its initial release, many erroneous annotations and user utterances have been identified and fixed in subsequent versions, i.e., MultiWOZ 2.1 (Eric et al., 2019) and MultiWOZ 2.2 (Zang et al., 2020).”
ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems (2020, methodology): “, 2017), MultiWOZ (Eric et al., 2019), DealOrNoDeal (Lewis et al.”
MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines (2020, background): “…the locations in the user and system utterances where slot values have been mentioned, to make the system efficient and generalizable to new slot values (Rastogi et al., 2017; Wu et al., 2019; Zhang et al., 2019; Rastogi et al., 2019; Xu and Hu, 2018; Zhou and Small, 2019; Gao et al., 2019).”
ConveRT: Efficient and Accurate Conversational Representations from Transformers (2019, methodology): “1 dataset (Eric et al., 2019), but we have opted for simple concatenation as an efficient solution for training on the large Reddit data.”
Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking (2021, background): “1 (Eric et al., 2019) and used in MultiWOZ2.”
Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking (2019, background): “1 (Eric et al., 2019) datasets, due to their annotation errors, ambiguity, and inconsistency.”
DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue (2020, background): “MULTIWOZ (Budzianowski et al., 2018; Eric et al., 2019) is a multi-domain dialogue dataset that contains 7 domains including restaurants, hotels and attractions.”
Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks (2020, methodology): “For multi-domain DST, slot-value pairs are extended to 1The official code https://github.com/jasonwu0731/trade-dst is used. domain-slot-value pairs for target (Ramadan, Budzianowski, and Gasic 2018; Gao et al. 2019; Wu et al. 2019).”
TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog (2020, background): “• The Multiwoz 2.1 (Eric et al., 2019) is an update to the Multiwoz (Budzianowski et al., 2018) dataset (written WoZ).”