Qwen2.5-Coder Technical Report

By Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu and 11 more. arXiv.org, 2024

1,385 citations recorded by the source corpus.

In this report, we introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. This series includes six models: Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/32B). As a code-specific model, Qwen2.5-Coder is built upon the Qwen2.5 architecture and continues pretrained on a vast corpus of over 5.5 trillion tokens. Through meticulous data cleaning, scalable synthetic data generation, and balanced data mixing, Qwen2.5-Coder demonstrates impressive code generation capabilities while retaining general and math skills. These models have been evaluated on a wide range of code-related tasks, achieving state-of-the-art (SOTA) performance across more than 10 benchmarks, including code generation, completion, reasoning, and repair, consistently outperforming larger models of the same model size. We believe that the release of the Qwen2.5-Coder series will advance research in code intelligence and, with its permissive licensing, support wider adoption by developers in real-world applications.

Subfields: Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/11dd2fc88747d20629f5aafab72ba649d93e969c

Citations in the corpus

• Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model (2025, neutral): “5-{7B, 32B} base models as our main foundation.”
• WebThinker: Empowering Large Reasoning Models with Deep Research Capability (2025, neutral): “5-Coder-32B-Instruct [19], QwQ-32B [50], Llama3.”
• Visual-RFT: Visual Reinforcement Fine-Tuning (2025, neutral): “Studies have explored improving LLMs’ performance in reasoning tasks such as solving mathematical problems [2, 22, 33, 43, 45] and coding [6, 8, 51, 54].”
• Qwen3 Technical Report (2025, neutral): “5-Coder (Hui et al., 2024) models to synthesize trillions of text tokens in different formats, including textbooks, question-answering, instructions, and code snippets, covering dozens of domains.”
• CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking (2024, unknown): “5-Coder-7B-Instruct (Hui et al., 2024), an instruction-tuned code generation model to judge whether a code snippet fully answers the corresponding query for 10k randomly sampled pairs from each dataset for 3 seeds.”
• When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection (2025, unknown): “5B-instruct have demonstrated superior performance in semantic search and retrieval tasks (Hui et al., 2024).”
• SweRank: Software Issue Localization with Code Ranking (2025, unknown): “A significant challenge when sourcing contrastive data directly from public GitHub repositories is”
• Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation (2025, unknown): “5-Coder-7B-Instruct [48]) and two Python-focused mono-lingual models ( Phi-1 [49] and Phi-1.”
• Agentic Software Issue Resolution with Large Language Models: A Survey (2025, unknown): “On the other hand, because of the complexity and broad applicability, issue resolution has become a key task for evaluating LLMs and software engineering (SE) agentic systems [11, 68, 184].”