DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence

By Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, Y. K. Li and 3 more. 2024

1,786 citations recorded by the source corpus.

The rapid development of large language models has revolutionized code intelligence in software development. However, the predominance of closed-source models has restricted extensive research and development. To address this, we introduce the DeepSeek-Coder series, a range of open-source code models with sizes from 1.3B to 33B, trained from scratch on 2 trillion tokens. These models are pre-trained on a high-quality project-level code corpus and employ a fill-in-the-blank task with a 16K window to enhance code generation and infilling. Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and unrestricted commercial use.

Subfields: Artificial Intelligence, Information Systems, Large Language Models, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jianfeng Gao
  2. Percy Liang
  3. Mohit Bansal
  4. Richard Socher
  5. Yoshua Bengio
  6. Quoc V. Le
  7. Luke S. Zettlemoyer
  8. Caiming Xiong

In Information Systems

  1. Chun Xia
  2. Lingming Zhang
  3. Yinlin Deng
  4. Ling-Ming Zhang
  5. Soren Dunn
  6. Armando Solar-Lezama
  7. Fei Huang
  8. Zhihong Shao

In Large Language Models

  1. Richard Socher
  2. Jianfeng Gao
  3. Lingming Zhang
  4. Chun Xia
  5. Armando Solar-Lezama
  6. Fei Huang
  7. Peiyi Wang
  8. Zhihong Shao

In Code Generation & Program Synthesis

  1. Armando Solar-Lezama
  2. Chun Xia
  3. Koushik Sen
  4. Lingming Zhang
  5. Daya Guo
  6. Rishabh Singh

Citations in the corpus