DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

By DeepSeek-AI, Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y. Wu, Yukun Li, Huazuo Gao and 30 more. arXiv.org, 2024

469 citations recorded by the source corpus.

We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks. Compared to DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in various aspects of code-related tasks, as well as reasoning and general capabilities. Additionally, DeepSeek-Coder-V2 expands its support for programming languages from 86 to 338, while extending the context length from 16K to 128K. In standard benchmark evaluations, DeepSeek-Coder-V2 achieves superior performance compared to closed-source models such as GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks.

Subfields: Molecular Biology

View Paper: https://www.semanticscholar.org/paper/2797cbda8c845504119b62ee25deb1500ec2dfaf

Citations in the corpus

• Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing (2024, unknown): “Specifically, we select 300K instances from M AGPIE -Pro and M AGPIE -Air-Filtered, yielding datasets M AGPIE -Pro-300K and M AGPIE -Air-300K-Filtered, respectively.”