Large Language Models Cannot Self-Correct Reasoning Yet

By Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, Denny Zhou. International Conference on Learning Representations, 2023

1,041 citations recorded by the source corpus.

Large Language Models (LLMs) have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content. A contemporary methodology, self-correction, has been proposed as a remedy to these issues. Building upon this premise, this paper critically examines the role and efficacy of self-correction within LLMs, shedding light on its true potential and limitations. Central to our investigation is the notion of intrinsic self-correction, whereby an LLM attempts to correct its initial responses based solely on its inherent capabilities, without the crutch of external feedback. In the context of reasoning, our research indicates that LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction. Drawing from these insights, we offer suggestions for future research and practical applications in this field.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/6d4bacb69923e1e94fb4de468b939ce6db32fb51

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Understanding the Effects of Iterative Prompting on Truthfulness (2024, background): “Despite prior explorations into iterative prompting, its efficacy and consequences are not unequivocally established, stirring debates and inquiries into its impact on model truthfulness[16].”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, neutral): “…has shown promise for achieving faithful and accurate reasoning, especially in iterative settings, certain studies (Stechly et al., 2023; Huang et al., 2023a; Valmeekam et al., 2023) question the effectiveness of the self-correction mechanism, pointing out that LLMs still struggle to…”
rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking (2025, neutral): “The community has actively explored various approaches, including self-correction [Huang et al., 2023, Kumar et al., 2024], self-reflection [Renze and Guven, 2024, Shinn et al., 2024], to explicitly train or prompt LLMs to develop such capability.”
Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models (2023, background): “This also enables scaling to more challenging environments, as it is difficult for LMs to improve their responses without external feedback (Huang et al., 2024).”
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs (2023, neutral): “A large body of recent work has focused on improving the reasoning capabilities of Large Language Models (LLMs) by imitating various human cognitive processes (Wang and Zhao, 2023; Park et al., 2023; Sumers et al., 2023; Ye et al., 2023).”
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations (2023, background): “…et al., 2023; Kaddour et al., 2023; Song et al.; Li et al., 2023a; Wang et al., 2023a; Chen et al., 2023; Zheng et al., 2023; Wang et al., 2023c), However, even the most advanced LLMs face challenges in complex multi-step mathematical reasoning problems (Lightman et al., 2023; Huang et al., 2023).”
Training Language Models to Self-Correct via Reinforcement Learning (2024, positive): “This self-correction capability has been shown to be severely lacking in current LLMs, especially in the absence of external input (also called intrinsic self-correction ) (Huang et al., 2023; Kamoi et al., 2024).”
Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training (2023, neutral): “…π θ : Use cross-entropy loss with trajectories’ tokens as target (2) Value network v θ : Mean squared error loss with trajectories’ temporal difference (TD) or Monte-Carlo (MC) based value estimation as target, and (3) ORM ˆ r θ : Mean squared error loss with trajectories’ final reward as target.”
Agentless: Demystifying LLM-based Software Engineering Agents (2024, background): “Existing agents struggle with the capability to perform self-reflection [69, 41].”