Dialogue in the Wild: Learning from a Deployed Role-Playing Game with Humans and Bots

By Kurt Shuster, Jack Urbanek, Emily Dinan, Arthur Szlam, Jason Aaron Edward Weston. 2021

15 citations recorded by the source corpus.

Much of NLP research has focused on crowd-sourced static datasets and the supervised learning paradigm of training once and then evaluating test performance. As argued in de Vries et al. (2020), crowdsourced data has the issues of lack of naturalness and relevance to real-world use cases, while the static dataset paradigm does not allow for a model to learn from its experiences of using language (Silver et al., 2013). In contrast, one might hope for machine learning systems that become more useful as they interact with people. In this work, we build and deploy a role-playing game, whereby human players converse with learning agents situated in an open-domain fantasy world. We show that by training models on the conversations they have with humans in the game the models progressively improve, as measured by automatic metrics and online engagement scores. This learning is shown to be more efﬁcient than crowdsourced data when applied to conversations with real users, as well as being far cheaper to collect.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, methodology): “The promise of such an approach is that the distribution of data will more closely match those organic users’ desires, rather than decided by the researchers themselves when creating datasets (Gabriel et al., 2020; Roller et al., 2020; Shuster et al., 2021b; Ouyang et al., 2022).”
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2021, methodology): “Notably, K2R hallucinates less than any model studied by Shuster et al. (2021a).”