AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts

By Tongshuang Sherry Wu, Michael Terry, Carrie J. Cai. 2021

679 citations recorded by the source corpus.

Although large language models (LLMs) have demonstrated impressive potential on simple tasks, their breadth of scope, lack of transparency, and insufficient controllability can make them less effective when assisting humans on more complex tasks. In response, we introduce the concept of Chaining LLM steps together, where the output of one step becomes the input for the next, thus aggregating the gains per step. We first define a set of LLM primitive operations useful for Chain construction, then present an interactive system where users can modify these Chains, along with their intermediate results, in a modular way. In a 20-person user study, we found that Chaining not only improved the quality of task outcomes, but also significantly enhanced system transparency, controllability, and sense of collaboration. Additionally, we saw that users developed new ways of interacting with LLMs through Chains: they leveraged sub-tasks to calibrate model expectations, compared and contrasted alternative strategies by observing parallel downstream effects, and debugged unexpected model outputs by “unit-testing” sub-components of a Chain. In two case studies, we further explore how LLM Chains may be used in future applications.

Subfields: Artificial Intelligence, Computer Science Applications, Information Systems, Prompting & Reasoning, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Percy Liang
  3. Luke S. Zettlemoyer
  4. Quoc V. Le
  5. Jianfeng Gao
  6. Yejin Choi
  7. Graham Neubig
  8. Yoshua Bengio

In Computer Science Applications

  1. Mihir Parmar
  2. Michael S. Bernstein
  3. Swaroop Mishra
  4. Chitta Baral
  5. Quoc V. Le
  6. Percy Liang
  7. Thomas Scialom
  8. Timo Schick

In Information Systems

  1. Percy Liang
  2. Quoc V. Le
  3. Denny Zhou
  4. Timo Schick
  5. Maria Lomeli
  6. Xinyun Chen
  7. Swaroop Mishra
  8. Chitta Baral

In Prompting & Reasoning

  1. Denny Zhou
  2. Swaroop Mishra
  3. Xinyun Chen
  4. Percy Liang
  5. Chitta Baral
  6. Jason Wei
  7. Tengyu Ma
  8. Xuezhi Wang

In LLM Agents & Tool Use

  1. Swaroop Mishra
  2. Mihir Parmar
  3. Chitta Baral
  4. Man Luo
  5. Percy Liang
  6. Qian Yang
  7. J.D. Zamfirescu-Pereira
  8. Bjoern Hartmann

Citations in the corpus