Kimi k1.5: Scaling Reinforcement Learning with LLMs

By Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao and 84 more. arXiv.org, 2025

1,006 citations recorded by the source corpus.

Language model pretraining with next token prediction has proved effective for scaling compute but is limited to the amount of available training data. Scaling reinforcement learning (RL) unlocks a new axis for the continued improvement of artificial intelligence, with the promise that large language models (LLMs) can scale their training data by learning to explore with rewards. However, prior published work has not produced competitive results. In light of this, we report on the training practice of Kimi k1.5, our latest multi-modal LLM trained with RL, including its RL training techniques, multi-modal data recipes, and infrastructure optimization. Long context scaling and improved policy optimization methods are key ingredients of our approach, which establishes a simplistic, effective RL framework without relying on more complex techniques such as Monte Carlo tree search, value functions, and process reward models. Notably, our system achieves state-of-the-art reasoning performance across multiple benchmarks and modalities -- e.g., 77.5 on AIME, 96.2 on MATH 500, 94-th percentile on Codeforces, 74.9 on MathVista -- matching OpenAI's o1. Moreover, we present effective long2short methods that use long-CoT techniques to improve short-CoT models, yielding state-of-the-art short-CoT reasoning results -- e.g., 60.8 on AIME, 94.6 on MATH500, 47.3 on LiveCodeBench -- outperforming existing short-CoT models such as GPT-4o and Claude Sonnet 3.5 by a large margin (up to +550%).

Subfields: Artificial Intelligence, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/668075792a7ab40457d92e09da28d35c879271c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Maosong Sun
2. Ning Ding
3. Zhiyuan Liu
4. Dahua Lin
5. Yuhang Cao
6. Yuan Yao
7. Zhilin Yang
8. Yuhang Zang

In Control and Systems Engineering

1. Yuan Yao
2. Dahua Lin
3. Francesco Orabona

Citations in the corpus

• Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025, neutral): “.5 [83]; O1-Pruner [57]; L1 [1]; Training [3]; Demystifying [104]; DAST [74]; MRT [68]; Stepwise [23]; SFT with Variable-Length CoT e.g. Distilling 2-1 [105]; C3oT [40]; TokenSkip [95]; CoT-Valve [59]; Self-Training [62]; Learn to Skip [53]; Token-Budget [34]; ReasoningOutput-based Latent e.g.…”
• R1-VL: Learning to Reason with Multimodal Large Language Models via Step-Wise Group Relative Policy Optimization (2025, neutral): “Recently, RL is increasingly adopted to enhance LLMs’ reasoning capabilities [4, 6, 12, 22, 32, 45], especially for mathematical problem solving.”
• Kimi-VL Technical Report (2025, positive): “5 (K. Team et al. 2025), we adopt a variant of online policy mirror descent as our RL algorithm, which iteratively refines the policy model π θ to improve its problem-solving accuracy.”
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale (2025, neutral): “However, the actual algorithm and key recipe for scalable RL training remains a myth, hidden from technical reports of existing reasoning models [1, 2, 8–11].”
• SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild (2025, neutral): “5 (Team et al., 2025), demonstrate remarkable abilities.”
• Process Reinforcement through Implicit Rewards (2025, positive): “However, successful applications of dense rewards in RL for LLMs are limited (Setlur et al., 2024), as current industry-leading models primarily depend on verifiable outcome rewards and have not yet demonstrated meaningful progress with dense rewards (DeepSeek-AI et al., 2025; Team et al., 2025).”
• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models (2025, positive): “…breakthrough on enhancing the reasoning capability of LLMs using RL with verifiable rewards (OpenAI, 2024a; Lambert et al., 2024; DeepSeek-AI et al., 2025; Team et al., 2025), drawing great attention in research community (Cui et al., 2025; Liu et al., 2025; Hu et al., 2025; He et al., 2025).”
• Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model (2025, neutral): “Theses models typically first acquire advanced reasoning patterns from existing reasoning models or high-quality human-labeled data [21, 22, 23, 24, 25] through techniques such as SFT distillation or other cold-start approaches.”
• Kimi K2: Open Agentic Intelligence (2025, neutral): “5 [36] as the foundation for K2.”
• Visual-RFT: Visual Reinforcement Fine-Tuning (2025, positive): “Re-inforcement Learning with Verifiable Rewards (RLVR) [4, 14, 39] is a novel training approach designed to enhance language models in tasks with objectively verifiable outcomes, such as math and coding.”
• Video-R1: Reinforcing Video Reasoning in MLLMs (2025, positive): “5 [21] and Skywork R1V [25], which apply RL to improve reasoning over image-text pairs.”
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (2025, neutral): “5 (Team et al., 2025), has significantly advanced the frontier of LLM capabilities, particularly in solving complex logical tasks involving mathematics and programming.”
• Group-in-Group Policy Optimization for LLM Agent Training (2025, neutral): “Most recent works have explored using RL to enhance the capabilities of reasoning and logical deduction in LLMs [55, 14].”
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs (2024, positive): “The OpenAI o1 model (OpenAI, 2024) and its replicas (Qwen, 2024; Guo et al., 2025; Kimi et al., 2025) exemplify the state-of-the-art in AI reasoning.”
• VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning (2025, neutral): “Our training data was compiled by integrating publicly available datasets [Du et al., 2025, Yang et al., 2025, Meng et al., 2025] with novel data collected from the web.”