A Repository of Conversational Datasets

By Matthew Henderson, Paweł Budzianowski, Iñigo Casanueva, Sam Coope, Daniela Gerz, Nikola Mrksic, Georgios P. Spithourakis, Pei-hao Su, Ivan Vulic, Tsung-Hsien Wen. 2019

97 citations recorded by the source corpus.

Progress in Machine Learning is often driven by the availability of large datasets, and consistent evaluation metrics for comparing modeling approaches. To this end, we present a repository of conversational datasets consisting of hundreds of millions of examples, and a standardised evaluation procedure for conversational response selection models using 1-of-100 accuracy. The repository contains scripts that allow researchers to reproduce the standard datasets, or to adapt the pre-processing and data filtering steps to their needs. We introduce and evaluate several competitive baselines for conversational response selection, whose implementations are shared in the repository, as well as a neural encoder model that is trained on the entire training set.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “Data available at github.com/PolyAI-LDN/conversational-datasets.”
Efficient Intent Detection with Dual Sentence Encoders (2020, methodology): “Note that e.g. ConveRT is much lighter in its design and is also pretrained more quickly than BERT (Henderson et al., 2019b): it relies on 6 Transfomer layers with embedding dimensionality of 512.”
ConveRT: Efficient and Accurate Conversational Representations from Transformers (2019, methodology): “We base all our (pre)training on the large Reddit conversational corpus (Henderson et al., 2019a) derived from 3.7B Reddit comments: it comprises 727M (input, response) pairs for single-context modeling – 654M pairs are reserved for training, the rest is used for testing.”
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, methodology): “Prior works use Reddit data for conversational response selection [5, 139, 54] and abstractive summarization [127, 65] We release all post IDs and their corresponding metadata.”
RedCaps: web-curated image-text data created by the people, for the people (2021, methodology): “Prior works in dialog modeling and text summarization have trained on Reddit comments [33, 37–40].”
Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization (2021, background): “With the availability of large-scale data, such as posts on social media or forums, scripts of movies or TV series, and datasets from collection or crowdsourcing [8, 16, 23], neural conversational models (NCMs) [44, 45] have developed rapidly and their performance has been improved steadily.”