Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

By Yang Yue, Zhiqin Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, Gao Huang. arXiv.org, 2025

919 citations recorded by the source corpus.

Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning performance of large language models (LLMs), particularly on mathematics and programming tasks. Similar to how traditional RL helps agents explore and learn new strategies, RLVR is believed to enable LLMs to continuously self-improve, thus acquiring novel reasoning abilities beyond those of the corresponding base models. In this study we critically examine the current state of RLVR by systematically probing the reasoning capability boundaries of RLVR-trained LLMs across various model families, RL algorithms, and math, coding, and visual reasoning benchmarks, using pass@k at large k values as the evaluation metric. Surprisingly, we find that the current training setup does not elicit fundamentally new reasoning patterns. While RLVR-trained models outperform their base models at small k (e.g., k = 1), the base models achieve a higher pass@k score when k is large. Coverage and perplexity analyses show that the observed reasoning abilities originate from and are bounded by the base model. Treating the base model as an upper bound, our quantitative analysis shows that six popular RLVR algorithms perform similarly and remain far from optimal in leveraging the potential of the base model. By contrast, we find that distillation can introduce new reasoning patterns from the teacher and genuinely expand the model's reasoning capabilities. Overall, our findings suggest that current RLVR methods have not yet realized the potential of RL to elicit truly novel reasoning abilities in LLMs. This highlights the need for improved RL paradigms, such as continual scaling and multi-turn agent-environment interaction, to unlock this potential.

View Paper: https://www.semanticscholar.org/paper/143e18bfd7c356592e7c1439738a3525d3e16279

Citations in the corpus

• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models (2025, unknown): “Basically, the policy is trading uncertainty (entropy) for rewards (Yue et al., 2025) in a predictable manner .”
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (2025, unknown): “Yue et al. [35] questioned whether reinforcement learning truly elicits novel reasoning patterns and shows pass@k of reasoning vs non-reasoning models converge to the same point.”
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, unknown): “However, despite much effort in RL algorithms [23, 24, 90, 109, 133, 152, 171, 183] and verifiers [82, 136, 143 .”
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning (2025, unknown): “…2025), cross-domain applications (Xue et al., 2025; Liu et al., 2025; Pan et al., 2025), and counterintuitive empirical insights (Wang et al., 2025; Yue et al., 2025a; Zhao et al., 2025), existing implementations directly train over all tokens with limited understanding of which tokens actually…”
• Are We Measuring Strategy or Phrasing? The Gap Between Surface- and Approach-Level Diversity in LLM Math Reasoning (2026, unknown): “…with verifiable rewards (RLVR), a dominant paradigm for post-training language models, reduces the diversity of a trained policy (Kirk et al., 2024; He et al., 2025; Yue et al., 2025; Wu and Choi, 2025), prompting diversity-aware RLVR methods that attempt to preserve diversity during training.”