Recipes for Building an Open-Domain Chatbot

By Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric Michael Smith and 2 more. 2020

1,145 citations recorded by the source corpus.

Building open-domain chatbots is a challenging area for machine learning research. While prior work has shown that scaling neural models in the number of parameters and the size of the data they are trained on gives improved results, we highlight other ingredients. Good conversation requires blended skills: providing engaging talking points, and displaying knowledge, empathy and personality appropriately, while maintaining a consistent persona. We show that large scale models can learn these skills when given appropriate training data and choice of generation strategy. We build variants of these recipes with 90M, 2.7B and 9.4B parameter models, and make our models and code publicly available. Human evaluations show our best models outperform existing approaches in multi-turn dialogue on engagingness and humanness measurements. We then discuss the limitations of this work by analyzing failure cases of our models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

LaMDA: Language Models for Dialog Applications (2022, methodology): “Dialog models [14, 15, 16], one of the most interesting applications of large language models, successfully take advantage of Transformers’ ability to represent long-term dependencies in text [17, 18].”
TruthfulQA: Measuring How Models Mimic Human Falsehoods (2021, background): “…Zellers et al., 2019), summarization (Gabriel et al., 2021; Maynez et al., 2020; Stiennon et al., 2020; Wang et al., 2020), conversational dialog (Shuster et al., 2021; Roller et al., 2021), and question answering (Dou et al., 2021; Krishna et al., 2021; Lewis et al., 2020; Logan IV et al., 2019).”
Detoxifying Language Models Risks Marginalizing Minority Voices (2021, background): “Unfortunately, because such corpora are too large to ﬁlter granu-larly (Roller et al., 2020), they inevitably contain so-called toxic examples: undesirable language such as expletives, slurs, or other offensive and threatening speech.”
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, methodology): “io Reddit (Baumgartner et al., 2020; Roller et al., 2020) using a next-word prediction objective.”
OPT: Open Pre-trained Transformer Language Models (2022, methodology): “The pre-training corpus contains a concatenation of datasets used in RoBERTa (Liu et al., 2019b), the Pile (Gao et al., 2021a), and PushShift.io Red-dit (Baumgartner et al., 2020; Roller et al., 2021).”
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, methodology): “plained by differences in model size and partially by overall differences in generation between the two classes of models (Adiwardana et al., 2020; Roller et al., 2021; Shuster et al., 2021).”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “For BlenderBot models, generation settings match Roller et al. (2020), using beam search with a beam size of 10, a minimum beam length of 20, and context n-gram blocking to avoid repetitiveness.”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “For dialog systems, Liu et al. (2019) Sheng et al. (2021) study biases relating to the personas assigned to the Blender (Roller et al., 2020) and DialoGPT (Zhang et al., 2019) dialog systems, presenting a tool to measure these biases, and demonstrating that a system’s persona can affect the level…”
Recipes for Safety in Open-domain Chatbots (2020, methodology): “We use the ACUTE eval (Li et al., 2019) method of evaluating conversational quality, as used for BlenderBot (Roller et al., 2020) and elsewhere.”
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “Altering the behavior of the generation method can dramatically influence human evaluations, while maintaining identical or near-identical perplexity (See et al., 2019; Welleck et al., 2020, 2019; Adiwardana et al., 2020; Roller et al., 2020).”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “An important benefit of E2E ConvAI models trained in this paradigm is that they can be adapted to new domains or taught new skills just by fine-tuning a pre-trained model on datasets of interest (e.g. Roller et al., 2020; Smith et al., 2020; Solaimon and Dennison, 2021).”
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion (2022, methodology): “…pre-training datasets for language-modeling and for dialogue, using the training method of Lewis et al. (2020a): pushshift.io Reddit We use a variant of Reddit discussions, which has also been used in several existing studies, particularly for training BlenderBot 1 and 2 (Roller et al., 2021).”
You Only Need One Model for Open-domain Question Answering (2021, background): “More importantly however, these language models often suffer from Hallucinations (Shuster et al., 2021; Maynez et al., 2020; Zhou et al., 2021; Zellers et al., 2019; Roller et al., 2021; Mielke et al., 2020), generating factually incorrect but plausiblesounding outputs.”
Retrieval Augmentation Reduces Hallucination in Conversation (2021, methodology): “Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge (Roller et al., 2020).”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “ach to generating responses (beyond question answering and information access discussed above) is to retrieve possible responses from a corpus (and then perhaps modifying it) [Henderson et al., 2017, Roller et al., 2020]. Here, the task is to rank possible responses with respect to their appropriate”
Autoregressive Entity Retrieval (2020, background): “…chat-bots and question answering systems, that are often equipped with retrieval components to surface speciﬁc KB entries (e.g., Wikipedia articles) to ﬁnd knowledge for sustaining a conversation or answering a question (Ferrucci, 2012; Chen et al., 2017; Lewis et al., 2020; Roller et al., 2020).”
Aligning Language Models with Preferences through f-divergence Minimization (2023, methodology): “…et al., 2020; Qin et al., 2022), Monte Carlo sampling techniques such as rejection sampling to sample from simple distributions like pGDC bin (Roller et al., 2021; Ziegler et al., 2022), and Quasi Rejection Sampling (QRS) (Eikema et al., 2022) or MCMC techniques (Miao et al., 2019; Goyal et…”
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, methodology): “We observe the best performance from BB3-175B across almost all metrics, including Good Response % and overall Rating compared to BB1, BB2, SeeKeR and variants of OPT-175B.”
A Simple Language Model for Task-Oriented Dialogue (2020, background): “They are usually trained end-to-end using large-scale data from social media [1, 42].”