CodeGen-Test: An Automatic Code Generation Model Integrating Program Test Information
By Maosheng Zhong, Gen Liu, HongWei Li, Jiangling Kuang, Jinshan Zeng, Mingwen Wang. 2022
16 citations recorded by the source corpus.
Automatic code generation is to generate the program code of the corresponding language according to the given natural language description. At present, the mainstream methods basically use neural network to encode natural language description, form an abstract syntax tree in the decoder, and then convert the abstract syntax tree into program code. Although the generated code conforms to specific syntax rules, two problems are still ignored: one is the lack of program testing, which is an indispensable step in the process of complete code implementation; Second, it only focuses on whether the generated code meets the syntax requirements, ignoring the more important requirement - functional requirements. This paper constructs a CodeGen test model for integrating code testing, that is, on the basis of generating code that meets the syntax requirements, add program testing steps, integrate program testing information, and iteratively generate code that meets the functional requirements of the program, so as to improve the quality of code generation. This paper evaluates the effect of CodeGen test model on a python data set "hearthstone legend". The experimental results show that this method can effectively improve the quality of generated code. Compared with the existing optimal model, CodeGen test model improves the Bleu value by 0.4%, Rougel value by 0.4%.
Subfields: Computer Science Applications, Control and Systems Engineering, Software, Code Generation & Program Synthesis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Science Applications
In Control and Systems Engineering
In Software
- Armando Solar-Lezama
- Koushik Sen
- Naman Jain
- Tianjun Zhang
- Ion Stoica
- Patrice Godefroid
- Rishabh Singh
- Gordon Fraser
In Code Generation & Program Synthesis
Citations in the corpus
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (2024, background): “…as an important application area for LLMs , with a proliferation of code-specific models (Chen et al., 2021; Austin et al., 2021; Li et al., 2022; Zhong et al., 2022; Allal et al., 2023; Li et al., 2023b; Roziere et al., 2023; Guo et al., 2024; Luo et al., 2023; Royzen et al., 2023; Wei et al.,…”
- CodeMind: A Framework to Challenge Large Language Models for Code Reasoning (2024, methodology): “Including test data in prompts has been a known practice to improve the performance of models in programming tasks (Chen et al., 2022; Zhong et al., 2022; Shi et al., 2022; Zhang et al., 2023).”