DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
By DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang and 186 more. Nature, 2025
5,477 citations recorded by the source corpus.
General reasoning represents a long-standing and formidable challenge in artificial intelligence (AI). Recent breakthroughs, exemplified by large language models (LLMs)1,2 and chain-of-thought (CoT) prompting3, have achieved considerable success on foundational reasoning tasks. However, this success is heavily contingent on extensive human-annotated demonstrations and the capabilities of models are still insufficient for more complex problems. Here we show that the reasoning abilities of LLMs can be incentivized through pure reinforcement learning (RL), obviating the need for human-labelled reasoning trajectories. The proposed RL framework facilitates the emergent development of advanced reasoning patterns, such as self-reflection, verification and dynamic strategy adaptation. Consequently, the trained model achieves superior performance on verifiable tasks such as mathematics, coding competitions and STEM fields, surpassing its counterparts trained through conventional supervised learning on human demonstrations. Moreover, the emergent reasoning patterns exhibited by these large-scale models can be systematically used to guide and enhance the reasoning capabilities of smaller models. A new artificial intelligence model, DeepSeek-R1, is introduced, demonstrating that the reasoning abilities of large language models can be incentivized through pure reinforcement learning, removing the need for human-annotated demonstrations.
Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning, Reinforcement Learning
View Paper: https://arxiv.org/abs/2501.12948
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Sergey Levine
- Yoshua Bengio
- Quoc V. Le
- Mohit Bansal
- Luke S. Zettlemoyer
- Ruslan Salakhutdinov
- Dacheng Tao
In Large Language Models
- Tiancheng Zhao
- Zhengxiao Du
- Jingren Zhou
- Vishrav Chaudhary
- Fei Huang
- Chujie Zheng
- Saksham Singhal
- Dale Schuurmans
In Prompting & Reasoning
In Reinforcement Learning
Citations in the corpus
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training (2025, positive): “Notably, recent findings (Jaech et al., 2024; DeepSeekAI et al., 2025) reveal “scaling laws” for inference-time compute, highlighting significant performance gains with increased computational resources.”
- Small Language Models are the Future of Agentic AI (2025, positive): “DeepSeek-R1-Distill is a family of reasoning models featuring 1.5-8bn sizes, trained on samples generated by DeepSeek-R1 [16].”
- Qwen3 Technical Report (2025, positive): “Notably, an increasing number of top-tier models (Meta-AI, 2025; Liu et al., 2024a; Guo et al., 2025; Yang et al., 2024b) are now being released as open-source, fostering broader research and innovation in artificial intelligence.”
- LIMO: Less is More for Reasoning (2025, positive): “Next, we subjected the filtered problems to a more rigorous evaluation using a stronger reasoning model, DeepSeek-R1-Distill-Qwen-32B (Guo et al., 2025).”
- R1-VL: Learning to Reason with Multimodal Large Language Models via Step-Wise Group Relative Policy Optimization (2025, positive): “Recently, RL is increasingly adopted to enhance LLMs’ reasoning capabilities [4, 6, 12, 22, 32, 45], especially for mathematical problem solving.”
- A-MEM: Agentic Memory for LLM Agents (2025, neutral): “Besides reporting experiment results with four additional metrics (ROUGE-L, ROUGE-2, METEOR, and SBERT Similarity), we also present experimental outcomes using different foundation models including DeepSeek-R1-32B [11], Claude 3.0 Haiku [2], and Claude 3.5 Haiku [3] in Appendix A.3.”
- VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model (2025, positive): “In parallel, R1-V introduced the GRPO method [46] from DeepSeek R1 [17] into VLM training, targeting object-counting tasks, and remarkably enabled a 3B model to outperform a 72B model.”
- Video-R1: Reinforcing Video Reasoning in MLLMs (2025, neutral): “Recent advancements in rule-based Reinforcement Learning (RL) [11] have significantly enhanced the reasoning capabilities of Large Language Models (LLMs) [10, 6].”
- Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025, positive): “Train-time scaling with RL [33] can significantly enhance the reasoning ability of LLMs.”
- Training Large Language Models to Reason in a Continuous Latent Space (2024, neutral): “One promising direction is pretraining LLMs with continuous thoughts, which may enable models to generalize more effectively across a wider range of reasoning scenarios.”
- Understanding R1-Zero-Like Training: A Critical Perspective (2025, neutral): “To investigate whether self-reflection behaviors are associated with model performance during the inference (acknowledging that self-reflection may improve exploration during training—a potential positive effect outside this section’s scope), we analyze questions that elicit at least one response wi”
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (2025, positive): “Following this, Deepseek-R1 (Guo et al., 2025) became the first open-weight model to match or surpass the performance of o1.”
- LLMs Get Lost In Multi-Turn Conversation (2025, neutral): “3-70B-Instruct, Llama 4 Scout) [23], AI2 OLMo-2-13B [56], Microsoft Phi-4 [1], Deepseek-R1 [24], and Cohere Command-A [15].”
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (2025, neutral): “[21], our work represents the first step towards this direction for the video domain.”