Think before you speak: Training Language Models With Pause Tokens

By Sachin Goyal, Z. Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, Vaishnavh Nagarajan. International Conference on Learning Representations, 2023

302 citations recorded by the source corpus.

Language models generate responses by producing a series of tokens in immediate succession: the $(K+1)^{th}$ token is an outcome of manipulating $K$ hidden vectors per layer, one vector per preceding token. What if instead we were to let the model manipulate say, $K+10$ hidden vectors, before it outputs the $(K+1)^{th}$ token? We operationalize this idea by performing training and inference on language models with a (learnable) $\textit{pause}$ token, a sequence of which is appended to the input prefix. We then delay extracting the model's outputs until the last pause token is seen, thereby allowing the model to process extra computation before committing to an answer. We empirically evaluate $\textit{pause-training}$ on decoder-only models of 1B and 130M parameters with causal pretraining on C4, and on downstream tasks covering reasoning, question-answering, general understanding and fact recall. Our main finding is that inference-time delays show gains when the model is both pre-trained and finetuned with delays. For the 1B model, we witness gains on 8 of 9 tasks, most prominently, a gain of $18\%$ EM score on the QA task of SQuAD, $8\%$ on CommonSenseQA and $1\%$ accuracy on the reasoning task of GSM8k. Our work raises a range of conceptual and practical future research questions on making delayed next-token prediction a widely applicable new paradigm.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/abe90a291e7cf567ce5c9012a692beeae153068d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (2025, unknown): “Another line of research focuses on compensating for the lack of supervised data by teaching models to think more effectively through supervised learning or reinforcement learning [22, 23, 24, 25, 26, 27].”
Training Large Language Models to Reason in a Continuous Latent Space (2024, unknown): “(4) Pause token (Goyal et al., 2023): The model is trained using only the question and answer, without a reasoning chain.”
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking (2024, background): “The left plot (a) shows the zero-shot accuracy on GSM8K, while the right plot (b) shows the zero-shot accuracy on CommonsenseQA, without any fine-tuning.”
Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs (2024, unknown): “Recent studies also explore latent space reasoning to mimic human cognition (Hao et al., 2024; Goyal et al., 2024).”
Better & Faster Large Language Models via Multi-token Prediction (2024, methodology): “To check the truth of this hypothesis, we augment the polynomial arithmetic task from Section 4.2 with a varying number of pause tokens (Goyal et al., 2023) inserted between the question and a token that denotes the beginning of the answer.”
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (2024, unknown): “Finally, several works have proposed methods for learning thought tokens in an unsupervised manner [12, 51], enabling models to more effectively utilize the additional test-time compute that comes with sampling longer sequences.”