Self-Consistency Improves Chain of Thought Reasoning in Language Models

By Xuezhi Wang, Jason Wei, Dale Schuurmans, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Denny Zhou. 2022

7,218 citations recorded by the source corpus.

Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer. Our extensive empirical evaluation shows that self-consistency boosts the performance of chain-of-thought prompting with a striking margin on a range of popular arithmetic and commonsense reasoning benchmarks, including GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%), StrategyQA (+6.4%) and ARC-challenge (+3.9%).

Subfields: Artificial Intelligence, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Jianfeng Gao
  4. Percy Liang
  5. Quoc V. Le
  6. Graham Neubig
  7. Yejin Choi
  8. Yoshua Bengio

In Prompting & Reasoning

  1. Denny Zhou
  2. Wenhu Chen
  3. Luke S. Zettlemoyer
  4. Jie Huang
  5. Swaroop Mishra
  6. Tushar Khot
  7. Peter Clark
  8. Xinyun Chen

Citations in the corpus