TALM: Tool Augmented Language Models

By Aaron Parisi, Yao Zhao, Noah Fiedel. 2022

207 citations recorded by the source corpus.

Transformer based language models (LMs) demonstrate increasing performance with scale across a wide variety of tasks. Scale alone however cannot enable models to solve tasks that require access to ephemeral, changing, or private data that was unavailable at training time. Many useful tasks may also benefit from LMs being able to access APIs that read or modify state. In this work, we present Tool Augmented Language Models (TALM), combining a text-only approach to augment language models with non-differentiable tools, and an iterative"self-play"technique to bootstrap performance starting from few tool demonstrations. TALM exhibits strong performance on both a knowledge-heavy QA task and a reasoning oriented math task with simple tools. At a given model scale, TALM significantly outperforms non-augmented LMs. We further demonstrate that TALM successfully performs out-of-distribution inferences on both QA and math tasks, where non-augmented LMs fail. Our results suggest that Tool Augmented Language Models are a promising direction to enrich LMs' capabilities, with less dependence on scale.

Subfields: Artificial Intelligence, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Percy Liang
  4. Jianfeng Gao
  5. Yoshua Bengio
  6. Quoc V. Le
  7. Graham Neubig
  8. Sergey Levine

In LLM Agents & Tool Use

  1. Karthik Narasimhan
  2. Aleksandra Faust
  3. Carl Vondrick
  4. Luke S. Zettlemoyer
  5. Mustafa Safdari
  6. D'idac Sur'is
  7. Izzeddin Gur
  8. Sachit Menon

Citations in the corpus