BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
By Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane and 8 more. 2022
297 citations recorded by the source corpus.
We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a long-term memory, and having been trained on a large number of user defined tasks. We release both the model weights and code, and have also deployed the model on a public web page to interact with organic users. This technical report describes how the model was built (architecture, model and training scheme), and details of its deployment, including safety mechanisms. Human evaluations show its superiority to existing open-domain dialogue agents, including its predecessors (Roller et al., 2021; Komeili et al., 2022). Finally, we detail our plan for continual learning using the data collected from deployment, which will also be publicly released. The goal of this research program is thus to enable the community to study ever-improving responsible agents that learn through interaction.
Subfields: Artificial Intelligence, Natural Language Processing, LLM Agents & Tool Use
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Jianfeng Gao
- Percy Liang
- Jason Aaron Edward Weston
- Yejin Choi
- Caiming Xiong
- Yoshua Bengio
In Natural Language Processing
- Jason Aaron Edward Weston
- Mohit Bansal
- Luke S. Zettlemoyer
- Jianfeng Gao
- Yejin Choi
- Emily Dinan
- Kurt Shuster
- Caiming Xiong
In LLM Agents & Tool Use
Citations in the corpus
- ReAct: Synergizing Reasoning and Acting in Language Models (2022, neutral): “In conversation modeling, chatbots like BlenderBot (Shuster et al., 2022b) and Sparrow (Glaese et al., 2022) and task-oriented dialogue systems like SimpleTOD (Hosseini-Asl et al., 2020) also train LMs to make decision about API calls.”
- OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, methodology): “Furthermore, adding 1% reasoning data results in the largest gains overall, beyond which, the gains start to reduce on MMLU, Cause-Effect Accuracy, Toxicity, and Dialogue (averaged over 0 and 5-shot).”
- When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels (2022, methodology): “The dataset DEMO is from the deployment of BlenderBot 3 (Shuster et al., 2022b) with responses verified by crowdworkers (Ju et al., 2022).”
- Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “This mechanism also allows for temporal generalisation beyond a static parametric model (Borgeaud et al., 2022; Lewis et al., 2020; Liška et al., 2022; Shuster et al., 2022a).”
- MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation (2022, background): “Meanwhile, there emerge several advanced models based on these corpora including generative ones (Zhang et al., 2020; Shuster et al., 2022; OpenAI, 2022) and retrieval-based ones (Wu et al.”
- CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society (2023, background): “These techniques underpin the impressive capabilities of recent dialogue LLMs [61, 68, 22, 6, 47, 10], which aim to simulate human-like conversations and provide personalized and interactive experiences for users, exhibiting the behavior of all three conversational AI agents [21].”
- MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback (2023, background): “For example, LLMs can connect with (1) the Internet to obtain latest information (Nakano et al., 2021; Shuster et al., 2022; Paranjape et al., 2023; Liu et al., 2023b); (2) the program interpreter to run the generated code (Chen et al., 2022; Gao et al., 2023; Drori et al., 2022; Pan et al., 2023;…”
- Regulating ChatGPT and other Large Generative AI Models (2023, background): “, MusicLM), while further alternatives are already in the pipeline [1-3].”
- GAIA: a benchmark for General AI Assistants (2023, methodology): “…et al., 2023; Chan et al., 2023; Talebirad and Nadiri, 2023); (iii) using single agent LLMs augmented with specific tools, such as Blender Bot 3 (Shuster et al., 2022), BOLAA (Liu et al., 2023b) and AssistGPT (Gao et al., 2023) extending LLMs with planning components, Socratic Models (Zeng et…”
- The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development (2023, background): “Using natural language to interact with technology has had a long research history [2], starting in the 1960s with pattern-matching approaches like Eliza [104], and continuing to today with state-of-the-art large language modelbased conversational systems [107] such as Meena [3] and BlenderBot [84].”
- Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, unknown): “Concretely, retrieved assets not only impose an additional velocity on particles for distribution initialization, but also help optimize the 2D diffusion model through low-rank adaptation.”
- A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, methodology): “ChatBot Xiaoice Microsoft 2018 [200] Xiaoice ChatBot Meena Google 2020 [201] Meena Blog ChatBot BlenderBot Meta 2022 [202] Blenderbot ChatBot ChatGPT OpenAI 2022 [10] ChatGPT ChatBot Alexa Amazon 2014 Amazon Alexa ChatBot Lex Amazon 2017 Amazon Lex Music AIVA Aiva Tech 2016 AIVA Music Jukebox OpenAI”