Demystifying Long Chain-of-Thought Reasoning in LLMs
By Edward Y. Chang, Yuxuan Tong, Morry Niu, Graham Neubig, Xiang Yue. arXiv.org, 2025
341 citations recorded by the source corpus.
Scaling inference compute enhances reasoning in large language models (LLMs), with long chains-of-thought (CoTs) enabling strategies like backtracking and error correction. Reinforcement learning (RL) has emerged as a crucial method for developing these capabilities, yet the conditions under which long CoTs emerge remain unclear, and RL training requires careful design choices. In this study, we systematically investigate the mechanics of long CoT reasoning, identifying the key factors that enable models to generate long CoT trajectories. Through extensive supervised fine-tuning (SFT) and RL experiments, we present four main findings: (1) While SFT is not strictly necessary, it simplifies training and improves efficiency; (2) Reasoning capabilities tend to emerge with increased training compute, but their development is not guaranteed, making reward shaping crucial for stabilizing CoT length growth; (3) Scaling verifiable reward signals is critical for RL. We find that leveraging noisy, web-extracted solutions with filtering mechanisms shows strong potential, particularly for out-of-distribution (OOD) tasks such as STEM reasoning; and (4) Core abilities like error correction are inherently present in base models, but incentivizing these skills effectively for complex tasks via RL demands significant compute, and measuring their emergence requires a nuanced approach. These insights provide practical guidance for optimizing training strategies to enhance long CoT reasoning in LLMs. Our code is available at: https://github.com/eddycmu/demystify-long-cot.
Subfields: Artificial Intelligence, Management Information Systems
View Paper: https://www.semanticscholar.org/paper/45e1c99a1c8935bf137c0b51a08a03ffb6821993
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- Understanding R1-Zero-Like Training: A Critical Perspective (2025, neutral): “A few prior studies (Liu et al., 2025b; Yeo et al., 2025) have suggested that there may not be Aha moment in open-source R1 replications because the base models they use already exhibit self-reflection keywords.”
- Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025, neutral): “.5 [83]; O1-Pruner [57]; L1 [1]; Training [3]; Demystifying [104]; DAST [74]; MRT [68]; Stepwise [23]; SFT with Variable-Length CoT e.g. Distilling 2-1 [105]; C3oT [40]; TokenSkip [95]; CoT-Valve [59]; Self-Training [62]; Learn to Skip [53]; Token-Budget [34]; ReasoningOutput-based Latent e.g.…”
- SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild (2025, neutral): “5-series models (Zeng et al., 2025a; Yeo et al., 2025; Xie et al., 2025; Hu et al., 2025; Yu et al., 2025), which, even as base models, exhibit strong instruction-following capabilities and display notable cognitive behaviors such as backtracking and verification from the beginning, as we will…”
- Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model (2025, positive): “We identify five representative reflection patterns (‘"wait,"‘, ‘"recheck"‘, ‘"retry"‘, ‘"alternatively,"‘, and ‘"however,"‘) and use this to determine whether a response is reflective, following a methodology similar to [14].”