OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement

By Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, Xiang Yue. 2024

279 citations recorded by the source corpus.

The introduction of large language models has significantly advanced code generation. However, open-source models often lack the execution capabilities and iterative refinement of advanced systems like the GPT-4 Code Interpreter. To address this, we introduce OpenCodeInterpreter, a family of open-source code systems designed for generating, executing, and iteratively refining code. Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions, OpenCodeInterpreter integrates execution and human feedback for dynamic code refinement. Our comprehensive evaluation of OpenCodeInterpreter across key benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6) with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap between open-source code generation models and proprietary systems like GPT-4 Code Interpreter.

Subfields: Artificial Intelligence, Software, LLM Agents & Tool Use, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Koushik Sen
  2. Xingyao Wang
  3. Hao Peng
  4. Yangyi Chen
  5. Armando Solar-Lezama
  6. Heng Ji
  7. Chun Xia
  8. Naman Jain

In Software

  1. Chun Xia
  2. Lingming Zhang
  3. Yinlin Deng
  4. Ling-Ming Zhang
  5. Soren Dunn
  6. Armando Solar-Lezama
  7. Koushik Sen
  8. Naman Jain

In Code Generation & Program Synthesis

  1. Armando Solar-Lezama
  2. Chun Xia
  3. Koushik Sen
  4. Hao Peng
  5. Lingming Zhang
  6. Rishabh Singh

Citations in the corpus