DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
By Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Jun-Mei Song, Mingchuan Zhang, Y. K. Li, Yu Wu, Daya Guo. 2024
7,648 citations recorded by the source corpus.
Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature. In this paper, we introduce DeepSeekMath 7B, which continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data. DeepSeekMath 7B has achieved an impressive score of 51.7% on the competition-level MATH benchmark without relying on external toolkits and voting techniques, approaching the performance level of Gemini-Ultra and GPT-4. Self-consistency over 64 samples from DeepSeekMath 7B achieves 60.9% on MATH. The mathematical reasoning capability of DeepSeekMath is attributed to two key factors: First, we harness the significant potential of publicly available web data through a meticulously engineered data selection pipeline. Second, we introduce Group Relative Policy Optimization (GRPO), a variant of Proximal Policy Optimization (PPO), that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO.
Subfields: Artificial Intelligence, Computational Theory and Mathematics, Large Language Models, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computational Theory and Mathematics
- Damai Dai
- Vitaliy Chiley
- Mohit Bansal
- Noah D. Goodman
- Yuan Yao
- Jingren Zhou
- Maosong Sun
- John P. Cunningham
In Large Language Models
In Prompting & Reasoning
Citations in the corpus
- Emerging Properties in Unified Multimodal Pretraining (2025, positive): “Inspired by DeepSeekMath [64], we first apply a lightweight topic selection process: LLMs are prompted to classify a small subset of documents, and the resulting labels are used to train fastText [34] classifiers for efficient large-scale inference.”
- WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct (2023, neutral): “…natural language processing (NLP) tasks, including open domain conversation (Ouyang et al., 2022; OpenAI, 2023; Touvron et al., 2023a), coding (Chen et al., 2021; Wang et al., 2021; Li et al., 2023b) and math (Taylor et al., 2022; Lewkowycz et al., 2022; Shao et al., 2024; Yang et al., 2024).”
- DanceGRPO: Unleashing GRPO on Visual Generation (2025, positive): “This work addresses these gaps by reformulating the sampling of diffusion models and rectified flows via Stochastic Differential Equations (SDEs) and applying Group Relative Policy Optimization (GRPO) [20, 21] to stabilize the training process.”
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (2025, positive): “Alternative solutions include Group Relative Policy Optimization (GRPO) (Shao et al., 2024), which eliminates the need for a critic model by estimating baselines from group scores, and RLOO (Ahmadian et al., 2024), which introduces a simplified REINFORCE-style (Williams, 1992) optimization…”
- Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs (2025, positive): “A notable breakthrough came with Deepseek-R1 (Guo et al., 2025; Jaech et al., 2024), which demonstrated that even a simplified version of PPO (Schulman et al., 2017) called GRPO (Shao et al., 2024) (2025) have begun to unpack these results, revealing that supervised fine-tuning with long, structured”
- Video-R1: Reinforcing Video Reasoning in MLLMs (2025, critical): “However, directly applying RL training with the Group Relative Policy Optimization (GRPO) algorithm [19] to video reasoning introduces two fundamental challenges: First, original GRPO lacks explicit reward signals for encouraging temporal reasoning in a video.”
- L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning (2025, neutral): “For GRPO training, we adopt the same hyperparameters as in DeepScaleR-1.”
- Flow-GRPO: Training Flow Matching Models via Online RL (2025, positive): “We begin by revisiting the core idea of GRPO [16] and adapting it to flow matching.”
- MAmmoTH2: Scaling Instructions from the Web (2024, neutral): “Open-source models like Llemma [Azerbayev et al., 2023], DeepSeek-Math [Shao et al., 2024], and Intern-Math [Ying et al., 2024] have surpassed MINERVA and Galactica on math benchmarks.”
- Process Reinforcement through Implicit Rewards (2025, positive): “We calculate the return of implicit process rewards and outcome rewards separately if both are available, since directly mixing their values may lead to numerical instability (Shao et al., 2024).”
- LIMO: Less is More for Reasoning (2025, neutral): “Two key developments have created conditions for reimagining reasoning in LLMs: (1) Knowledge foundation revolution : Modern foundation models now incorporate unprecedented amounts of mathematical content during pre-training (Qwen et al., 2025; Yang et al., 2024; Wang et al., 2024).”
- The Lessons of Developing Process Reward Models in Mathematical Reasoning (2025, neutral): “5-Math-7B-Instruct and the benchmark P ROCESS B ENCH (Zheng et al., 2024) across multiple PRMs (see Table 6 and Table 7 for details).”
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (2025, positive): “Specifically, we followed the DeepSeek-R1-Zero pipeline, training a base MLLM using GRPO [18], with output format constraints dictated by the following system prompt: • A conversation between User and Assistant.”
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (2025, positive): “Recently, Reinforcement Learning with Verifiable Rewards (RLVR) has gained significant traction as a method to improve the reasoning capabilities of LLMs in domains such as mathematics and programming (Lambert et al., 2024; Shao et al., 2024).”
- Training Large Language Models to Reason in a Continuous Latent Space (2024, neutral): “…prompting LLMs (Wei et al., 2022; Khot et al., 2022; Zhou et al., 2022), or training LLMs to generate reasoning chains, either with supervised finetuning (Yue et al., 2023; Yu et al., 2023) or reinforcement learning (Wang et al., 2024; Havrilla et al., 2024; Shao et al., 2024; Yu et al., 2024a).”
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning (2025, neutral): “Recently, numerous studies have shown that long-form reasoning [60] and scalable reinforcement learning [42] can significantly enhance the ability of large language models (LLMs) to solve complex problems [23; 17].”
- To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning (2024, neutral): “MMLU and MMLU Pro show gains from adding CoT, but because these datasets are so broad, they defy simple characterization.”