Show Your Work: Scratchpads for Intermediate Computation with Language Models
By Maxwell Nye, Anders Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, D. Luan and 2 more. 2021
1,075 citations recorded by the source corpus.
Large pre-trained language models perform remarkably well on tasks that can be done"in one pass", such as generating realistic text or synthesizing computer programs. However, they struggle with tasks that require unbounded multi-step computation, such as adding integers or executing programs. Surprisingly, we find that these same models are able to perform complex multi-step computations -- even in the few-shot regime -- when asked to perform the operation"step by step", showing the results of intermediate computations. In particular, we train transformers to perform multi-step computations by asking them to emit intermediate computation steps into a"scratchpad". On a series of increasingly complex tasks ranging from long addition to the execution of arbitrary programs, we show that scratchpads dramatically improve the ability of language models to perform multi-step computations.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Percy Liang
- Jianfeng Gao
- Quoc V. Le
- Yejin Choi
- Graham Neubig
- Yoshua Bengio
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Ming-Hsuan Yang
- Andrew Zisserman
- Bernt Schiele
In Natural Language Processing
- Mohit Bansal
- Luke S. Zettlemoyer
- Yejin Choi
- Graham Neubig
- Hannaneh Hajishirzi
- Percy Liang
- Noah A. Smith
- Jianfeng Gao
In Prompting & Reasoning
- Luke S. Zettlemoyer
- Denny Zhou
- Hannaneh Hajishirzi
- Yejin Choi
- Wenhu Chen
- Swaroop Mishra
- Jason Wei
- Tushar Khot
Citations in the corpus
- PaLM: Scaling Language Modeling with Pathways (2022, background): “Several recent papers have shown that large language models can achieve significant accuracy improvements by generating intermediate reasoning steps before generating the final answer (Nye et al., 2021; Cobbe et al., 2021; Wei et al., 2022b).”
- Constitutional AI: Harmlessness from AI Feedback (2022, methodology): “As we discuss below, our constitutional approach leverages chain-of-thought reasoning [Nye et al., 2021, Wei et al., 2022] to make decision making more legible.”
- Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “, 2019) and program synthesis/execution (Chen et al., 2021; Nye et al., 2021).”
- ReAct: Synergizing Reasoning and Acting in Language Models (2022, positive): “Similar approaches like Scratchpad (Nye et al., 2021), which finetunes a LM on intermediate computation steps, also demonstrate improvement on multi-step computation problems.”
- Solving Quantitative Reasoning Problems with Language Models (2022, background): “Prompting language models using scratchpad [15] or chain-of-thought [16] solutions can lead them to generate step-by-step solutions to unseen problems.”
- Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, methodology): “Nye et al. (2021) improve the logical consistency of samples from a System 1 model by adding a System 2-inspired logical reasoning module.”
- OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, methodology): “To solve these challenging tasks, recent work used different prompting methods which included a rationale with the final answer in the form of a scratchpad for arithmetic and logical reasoning (Nye et al., 2021), provided chain-of-thought prompts (Wei et al.”
- Emergent Abilities of Large Language Models (2022, background): “Reasoning tasks, especially those involving multiple steps, have been challenging for language models and NLP models more broadly (Rae et al., 2021; Bommasani et al., 2021; Nye et al., 2021).”
- GPT-NeoX-20B: An Open-Source Autoregressive Language Model (2022, methodology): “Prompt programming or finetuning can be used to leverage this fact and force a LLM to execute more transparent algorithms, such as splitting problems into steps or explicitly writing an “internal monologue” (Soares, 2021; Gao et al., 2021a; Nye et al., 2021).”
- CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis (2022, background): “Prompting pre-trained language models with intermediate information to improve task performance has attracted interest (Nye et al., 2021; Wei et al., 2022).”
- Augmented Language Models: a Survey (2023, background): “Note however that Nye et al. (2021) also perform experiments in the few-shot regime.”
- Large Language Models Are Human-Level Prompt Engineers (2022, background): “Early works (Nye et al., 2021; Betz et al., 2021; Wei et al., 2022b) on chain-of-thought used fine-tuning or in-context learning to get LLMs to show their work for such problems.”
- Scaling Instruction-Finetuned Language Models (2022, methodology): “Prior work either finetunes language models on a single reasoning dataset (Ling et al., 2017; Camburu et al., 2018; Cobbe et al., 2021; Nye et al., 2021; Zelikman et al., 2022, inter alia ) or focuses on models of substantially smaller scale (Ling et al., 2017; Camburu et al., 2018; Rajani et al.,…”
- Measuring Progress on Scalable Oversight for Large Language Models (2022, methodology): “Model (best-of-20 CoT): Here, we prompt the model with a chain-of-thought-style input before asking them to answer (Nye et al., 2021; Wei et al., 2022).”
- Large Language Models are Zero-Shot Reasoners (2022, background): “…increased by making them produce step-by-step reasoning, either by fine-tuning [Rajani et al., 2019, Cobbe et al., 2021, Zelikman et al., 2022, Nye et al., 2022] or few-shot prompting [Wei et al., 2022, Wang et al., 2022, Chowdhery et al., 2022] (See Table 6 for summary of related work).”
- Can language models learn from explanations in context? (2022, background): “Other recent work has shown that models can benefit from examples that decompose the reasoning process leading to an answer (Wei et al., 2022), especially when they are augmented with external scratchpads to store intermediate computations (Nye et al., 2021; Recchia, 2021).”