DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue

By Shikib Mehri, Mihail Eric, Dilek Z. Hakkani-Tür. 2020

148 citations recorded by the source corpus.

A long-standing goal of task-oriented dialogue research is the ability to flexibly adapt dialogue models to new domains. To progress research in this direction, we introduce DialoGLUE (Dialogue Language Understanding Evaluation), a public benchmark consisting of 7 task-oriented dialogue datasets covering 4 distinct natural language understanding tasks, designed to encourage dialogue research in representation-based transfer, domain adaptation, and sample-efficient task learning. We release several strong baseline models, demonstrating performance improvements over a vanilla BERT architecture and state-of-the-art results on 5 out of 7 tasks, by pre-training on a large open-domain dialogue corpus and task-adaptive self-supervised training. Through the DialoGLUE benchmark, the baseline methods, and our evaluation scripts, we hope to facilitate progress towards the goal of developing more general task-oriented dialogue models.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jianfeng Gao
  2. Ivan Vulic
  3. Mohit Bansal
  4. Nan Duan
  5. Ming Zhou
  6. Weizhu Chen
  7. Jason Aaron Edward Weston
  8. Richard Socher

In Evaluation & Benchmarks

  1. Ivan Vulic
  2. Fei Huang
  3. Matthew Henderson
  4. Yu Yan
  5. Weizhen Qi
  6. Ajit Rajasekharan
  7. Xiao-Dan Zhu
  8. Luo Si

In Dialogue & Conversational AI

  1. Matthew Henderson
  2. Ajit Rajasekharan
  3. Pei-hao Su
  4. Sivanesan Sangeetha
  5. Katikapalli Subramanyam Kalyan
  6. Daxin Jiang

Citations in the corpus