Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering

By Tal Ridnik, Dedy Kredo, Itamar Friedman. 2024

136 citations recorded by the source corpus.

Code generation problems differ from common natural language problems - they require matching the exact syntax of the target language, identifying happy paths and edge cases, paying attention to numerous small details in the problem spec, and addressing other code-specific issues and requirements. Hence, many of the optimizations and tricks that have been successful in natural language generation may not be effective for code tasks. In this work, we propose a new approach to code generation by LLMs, which we call AlphaCodium - a test-based, multi-stage, code-oriented iterative flow, that improves the performances of LLMs on code problems. We tested AlphaCodium on a challenging code generation dataset called CodeContests, which includes competitive programming problems from platforms such as Codeforces. The proposed flow consistently and significantly improves results. On the validation set, for example, GPT-4 accuracy (pass@5) increased from 19% with a single well-designed direct prompt to 44% with the AlphaCodium flow. Many of the principles and best practices acquired in this work, we believe, are broadly applicable to general code generation tasks. Full implementation is available at: https://github.com/Codium-ai/AlphaCodium

Subfields: Artificial Intelligence, Information Systems, Software, Large Language Models, Prompting & Reasoning, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Software

In Large Language Models

In Code Generation & Program Synthesis

Citations in the corpus

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (2024, background): “…ALGO (Zhang et al., 2023d), Parsel (Zelikman et al., 2022), code cleaning (Jain et al., 2023), code explainations (Li et al., 2023a), analogical reasoning (Yasunaga et al., 2023), and AlphaCodium (Ridnik et al., 2024) have been pushing the boundaries of what is possible with LLMs in this domain.”
AlgoSimBench: Identifying Algorithmically Similar Problems for Competitive Programming (2025, unknown): “Unlike previous methods [Xiong et al. , 2026; Shi et al. , 2022], ASM does not require any information from human-generated oracle solutions, and yet achieves comparable performance.”