Language Model Cascades

By David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Narain Sohl-Dickstein and 2 more. arXiv.org, 2022

119 citations recorded by the source corpus.

Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are probabilistic models, and may be expressed in the language of graphical models with random variables whose values are complex data types such as strings. Cases with control flow and dynamic structure require techniques from probabilistic programming, which allow implementing disparate model structures and inference strategies in a unified language. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use. We refer to the resulting programs as language model cascades.

Subfields: Artificial Intelligence, Natural Language Processing, Prompting & Reasoning

View Paper: https://arxiv.org/abs/2207.10342

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Prompting & Reasoning

Citations in the corpus

Improving alignment of dialogue agents via targeted human judgements (2022, background): “We believe these limitations are best addressed via multistep reasoning (Creswell et al., 2022; Dohan et al., 2022; Lewkowycz et al., 2022), with the further benefit of producing interpretable reasoning traces.”
Aligning Language Models with Preferences through f-divergence Minimization (2023, methodology): “, 2022) or selected samples from the model (Zelikman et al., 2022; Scheurer et al., 2022; Dohan et al., 2022), and fine-tuning the language model using reinforcement learning with a learned reward function that approximates human feedback (Reinforcement Learning from Human Feedback or RLHF; Ziegler ”
PAL: Program-aided Language Models (2022, background): “In fact, even when fine-tuning a PaLM-based model on 164B tokens of explicit mathematical content, its two most common failures are reportedly “incorrect reasoning” and “incorrect calculation” (Lewkowycz et al., 2022).”
PEER: A Collaborative Language Model (2022, methodology): “…of writing coherent pieces of text by iteratively performing small updates has some similarity with recent approaches like chain-of-thought prompting (Wei et al., 2022b; Dohan et al., 2022) and document sketching (Wu et al., 2021), that also break down a complex task into multiple smaller steps.”
RL with KL penalties is better viewed as Bayesian inference (2022, background): “Dohan et al. (2022) provide a unifying perspective on multi-step generation as latent variable modelling. the RL objective for LM alignment is ﬂawed: it does not care about preserving distributional properties of an LM and will always penalise the LM for putting any probability mass on non-optimal…”
Decomposed Prompting: A Modular Approach for Solving Complex Tasks (2022, methodology): “Formally, performing inference involves finding the best answer A to a new query Q, which in the simplest form of prompting involves computing the MAP answer using the LLMs predictive distribution for A, i.e., Â = argmaxA p(A | D,Q, θ) (Dohan et al., 2022).”
Parsel: A Unified Natural Language Framework for Algorithmic Reasoning (2022, background): “Unlike LM Cascades (Dohan et al., 2022), we primarily focus on the constrained generation of programs, instead of leveraging language models as functions within a particular program.”
Language Models can Solve Computer Tasks (2023, background): “This discovery has led to the emergence of a new line of research where reasoning capabilities are used to address tasks beyond reasoning [61, 22], or enhance reasoning proficiency [26, 28, 54, 29, 62, 40, 58, 51, 12].”
Large Language Models Can Be Easily Distracted by Irrelevant Context (2023, background): “…al., 2021; Wei et al., 2022; Suzgun et al., 2022; Shi et al., 2022b, inter alia ), problem decomposition (Zhou et al., 2022; Drozdov et al., 2022; Dohan et al., 2022; Khot et al., 2022; Press et al., 2022, inter alia ), generating programs (Austin et al., 2021; Chowdhery et al., 2022; Gao et…”
Training Language Models with Language Feedback at Scale (2023, background): “Dohan et al. (2022) further argues that the process of generating output through multiple rounds of interaction between prompted LMs and other agents (e.g. humans providing language feedback) can be seen as executing probabilistic programs.”
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP (2022, methodology): “…REDICT ), using them to systematically unify techniques from the retrieval-augmented NLP and the in-context learning literatures (Lee et al., 2019; Khattab et al., 2021a; Anan-tha et al., 2020; Gao et al., 2022; Izacard et al., 2022; Dohan et al., 2022; Zelikman et al., 2022; Zhang et al., 2022).”
Towards Reasoning in Large Language Models: A Survey (2022, background): “Lewkowycz et al. (2022); Taylor et al. (2022) find that LLMs trained on datasets containing scientific and mathematical data can achieve better performance on reasoning tasks like quantitative reasoning problems when using CoT prompting 3 .”
Solving math word problems with process- and outcome-based feedback (2022, unknown): “We ﬁnd that the 5-shot prompted policy alone (Few-shot) achieves 41.5% ﬁnal-answer error with 𝐾 = 96 majority voting (and 77.7% error when using a single sample).”
Cognitive Architectures for Language Agents (2023, methodology): “Prompting methods thus define a sequence of productions.”
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (2023, background): “Future work may benefit from the way variables are defined in Op-tuna (Akiba et al., 2019) and in the conceptual framework of Dohan et al. (2022) to allow expressing these.”
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (2022, background): “Recently, chain-of-thought (CoT) prompting has enabled large language models (LLMs) to perform logical reasoning tasks with impressive accuracy (Wei et al., 2022; Chowdhery et al., 2022; Lewkowycz et al., 2022).”
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model (2022, background): “This is an example of general architecture that has been referred to as a language model cascade (Dohan et al., 2022), a framework that generalizes earlier work on prompt chaining (Wu et al.”