Toolformer: Language Models Can Teach Themselves to Use Tools
By Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke S. Zettlemoyer, Nicola Cancedda, Thomas Scialom. Neural Information Processing Systems, 2023
4,753 citations recorded by the source corpus.
Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel. In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q\&A system, two different search engines, a translation system, and a calendar. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.
Subfields: Artificial Intelligence, Large Language Models, LLM Agents & Tool Use
View Paper: https://arxiv.org/abs/2302.04761
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Graham Neubig
- Quoc V. Le
- Yejin Choi
- Caiming Xiong
In Large Language Models
- Jianfeng Gao
- Luke S. Zettlemoyer
- Zhiyuan Liu
- Graham Neubig
- Weizhu Chen
- Hinrich Schütze
- Hannaneh Hajishirzi
- Danqi Chen
In LLM Agents & Tool Use
- Stephen Marcus McAleer
- Pierre Baldi
- Geunwoo Kim
- Aleksandra Faust
- Izzeddin Gur
- Panupong Pasupat
- Mustafa Safdari
- Kevin Qinghong Lin
Citations in the corpus
- Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, methodology): “The approach devised in Toolformer (Schick et al., 2023) entails the sampling of millions”
- MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action (2023, background): “Recent studies [11,24, 26,34] have attempted to merge the action and reasoning phases to enhance LLMs’ capabilities in solving complicated tasks that require advanced planning and reasoning.”
- ViperGPT: Visual Inference via Python Execution for Reasoning (2023, methodology): “In NLP, they have been aimed at using external tools [19,23], or for structured reasoning using Codex [6, 7, 9, 17, 28].”
- GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models (2023, background): “Moreover, recent studies reveal the potential of LLMs to program and control other digital tools, such as APIs, search engines, and even other generative AI systems (Schick et al., 2023; Mialon et al., 2023; Chase, 2022).”
- Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation (2024, methodology): “We can explore process supervision methods like those used in PRM-800K (Lightman et al., 2023), or investigate distillation techniques on successful trajectories, similar to approaches in ToolFormer (Schick et al., 2024) and DSPy (Khattab et al., 2023).”
- Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks (2022, background): “Another line of work related to ours is Tool-use in transformer models (Schick et al., 2023; Paranjape et al., 2023).”
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations (2023, background): “…commonplace in chat assistant applications, exhibiting excellent linguistic abilities (Brown et al., 2020; Anil et al., 2023; Touvron et al., 2023), commonsense reasoning (Wei et al., 2022b; Yao et al., 2023), and general tool use (Schick et al., 2023; Cai et al., 2023) among other capabilities.”
- GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher (2023, background): “…2023a) have demonstrated their advanced capability to perform innovative applications, ranging from tool utilization, supplementing human evaluations, to stimulating human interactive behaviors (Bubeck et al., 2023; Schick et al., 2024; Chiang & Lee, 2023; Park et al., 2023; Jiao et al., 2023).”
- Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System (2023, background): “Reasoning is defined as breaking down complex tasks into simpler subtasks that the LM can solve more easily by itself or with the help of tools [9,15,13], while tools are external modules that the LM can call to augment its context.”
- A survey on large language model based autonomous agents (2023, neutral): “Motivated by this capability, a burgeoning trend has emerged in recent years (see Figure 1 for the growth trend of this field), wherein LLMs are harnessed as core orchestrators in the creation of autonomous agents [19, 125, 123, 115, 119, 161].”
- Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (2023, methodology): “Tool 1 (Schick et al., 2023; Qin et al., 2023) or retrieval augmentation (Mallen et al., 2022; Shi et al., 2023b; Ram et al., 2023) has emerged as a promising solution by providing external information as new evidence to LLMs, such as ChatGPT Plugins and New Bing.”
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions (2024, background): “Modern large language models (LLMs) are no longer envisioned as simple autocomplete systems—they could instead empower agentic applications such as web agents, email secretaries, virtual assistants, and more (Nakano et al., 2021; Parisi et al., 2022; Schick et al., 2024; Shen et al., 2024).”
- Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models (2023, methodology): “Alternatively, to pure decision-making environments, the reasoning and practical abilities of LMs have been enhanced by providing access to external tools, such as APIs, search engines, calculators, and other models (Schick et al., 2023; Shen et al., 2023; Sur´ıs et al., 2023).”
- Reflexion: language agents with verbal reinforcement learning (2023, background): “Recent works such as ReAct [30], SayCan [1], Toolformer [22], HuggingGPT [23], generative agents [19], and WebGPT [17] have demonstrated the feasibility of autonomous decision-making agents that are built on top of a large language model (LLM) core.”
- A Survey of Large Language Models (2023, neutral): “, search engine [70], calculator [69], and compiler [68], to enhance the performance of LLMs on several specific tasks.”
- Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs (2023, methodology): “Drawing inspirations from Toolformer [65], this study utilizes LLMs as an interface to bridge the natural language commands and GNNs.”