Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

By Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke S. Zettlemoyer, Huan Sun. 2022

397 citations recorded by the source corpus.

Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). CoT explicitly encourages the LLM to generate intermediate rationales for solving a problem, by providing a series of reasoning steps in the demonstrations. Despite its success, there is still little understanding of what makes CoT prompting effective and which aspects of the demonstrated reasoning steps contribute to its performance. In this paper, we show that CoT reasoning is possible even with invalid demonstrations - prompting with invalid reasoning steps can achieve over 80-90% of the performance obtained using CoT under various metrics, while still generating coherent lines of reasoning during inference. Further experiments show that other aspects of the rationales, such as being relevant to the query and correctly ordering the reasoning steps, are much more important for effective CoT reasoning. Overall, these findings both deepen our understanding of CoT prompting, and open up new questions regarding LLMs’ capability to learn to reason in context.

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Large Language Models Can Self-Improve (2022, neutral): “…intermediate steps, and show it can help language models better solve complex and multi-step reasoning tasks, with recent study (Wang et al., 2022a) analyzing the relevant contents and correct reasoning order being the most crucial factor of the success of Chain-of-Thought…”
Training Large Language Models to Reason in a Continuous Latent Space (2024, neutral): “This phenomenon is known as the unfaithfulness of CoT reasoning (Wang et al., 2022; Turpin et al., 2024).”
Think before you speak: Training Language Models With Pause Tokens (2023, background): “CoT prompting has been shown to significantly improve the reasoning capabilities of large language models (Wei et al., 2022; Nye et al., 2021; Lanchantin et al., 2023; Suzgun et al., 2023; Zelikman et al., 2022; Zhou et al., 2023; Wang et al., 2023b; Yao et al., 2023).”
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems (2024, methodology): “…of generating thought steps extends to zero-shot CoT prompting (Kojima et al., 2022), where the LLMs are only simply instructed with the prompt “let’s think step by step”, and to even using incorrect reasoning steps in the few-shot examples (Wang et al., 2022a; Madaan & Yazdanbakhsh, 2022).”
RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback (2023, background): “It is worth noting that in-context learning is still an important research area that is not fully understood (Min et al., 2022; Wang et al., 2022a).”
Toward expert-level medical question answering with large language models (2025, neutral): “56 to improve performance on multiple-choice benchmarks by sampling multiple explanations and answers from the model.”
Large Language Models Can Be Easily Distracted by Irrelevant Context (2023, background): “This means that in addition to improving model performance on clean arithmetic reasoning tasks (Wang et al., 2022c), SC also substantially reduces the distractibility of large language models to irrelevant context.”
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning (2024, background): “…can provide human-readable explanations of how problems are solved (Joshi et al., 2023; Lanham et al., 2023), but most frequently it is invoked to improve an LLM’s ability to answer complex questions via intermediate computation (Madaan & Yazdanbakhsh, 2022; Wang et al., 2023a; Dziri et al., 2023).”
Towards Reasoning in Large Language Models: A Survey (2022, background): “In addition, Wang et al. (2022b) conduct ablation experiments on CoT and find that LLMs may also perform reasoning while prompting with invalid rationals.”
Explainability for Large Language Models: A Survey (2023, background): “The more consistent the multiple responses are, the higher the estimated confidence in the answer is.”
RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback (2023, neutral): “It’s interesting to note that in-context learning is still an important research area that is not fully understood (Min et al., 2022; Wang et al., 2022a).”
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning (2023, methodology): “More recently, (Chen et al., 2023; Wang et al., 2023e) have proposed to tackle more challenging college-level science and math problems.”
Larger language models do in-context learning differently (2023, background): “Along the same lines, Madaan & Yazdanbakhsh (2022) and Wang et al. (2022) show that for chain-of-thought prompting (Wei et al., 2022c), logically-incorrect prompts do not hurt performance on multi-step reasoning tasks.”