Language Models Can Teach Themselves to Program Better

By Patrick M. Haluptzok, Matthew Bowers, Adam Tauman Kalai. 2022

103 citations recorded by the source corpus.

Recent Language Models (LMs) achieve breakthrough performance in code generation when trained on human-authored problems, even solving some competitive-programming problems. Self-play has proven useful in games such as Go, and thus it is natural to ask whether LMs can generate their own instructive programming problems to improve their performance. We show that it is possible for an LM to synthesize programming problems and solutions, which are filtered for correctness by a Python interpreter. The LM's performance is then seen to improve when it is fine-tuned on its own synthetic problems and verified solutions; thus the model 'improves itself' using the Python interpreter. Problems are specified formally as programming puzzles [Schuster et al., 2021], a code-based problem format where solutions can easily be verified for correctness by execution. In experiments on publicly-available LMs, test accuracy more than doubles. This work demonstrates the potential for code LMs, with an interpreter, to generate instructive problems and improve their own performance.

Subfields: Artificial Intelligence, Information Systems, Large Language Models, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Large Language Models

In Code Generation & Program Synthesis

Citations in the corpus

Execution-Based Evaluation for Open-Domain Code Generation (2022, background): “Moreover, many datasets focus on programming challenges (Li et al., 2022; Haluptzok et al., 2022) and are often unrepresentative of coding queries asked in practical scenarios.”
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking (2024, background): “Zelikman et al. (2023b) shows how a bootstrapping loop can be implemented where a model repeatedly improves a code-improver using the same code-improver and Haluptzok et al. (2023) shows how language models can bootstrap their programming ability with programming puzzles Schuster et al. (2021).”
Will we run out of data? Limits of LLM scaling based on human-generated data (2022, background): “On the other hand, training on synthetic data has shown much promise in domains where model outputs are relatively easy to verify, such as mathematics, programming, and games (Yang et al., 2023; Liu et al., 2023; Haluptzok et al., 2023).”
Reflexion: an autonomous agent with dynamic memory and self-reflection (2023, background): “(Huang et al., 2022a; Haluptzok et al., 2022) use self-generated solutions to fine-tune an LLM to improve performance without access to a labeled dataset.”
LLM-Assisted Code Cleaning For Training Accurate Code Generators (2023, background): “Haluptzok et al. (2023) similarly generates programming puzzles and corresponding solutions from language models.”
Mathematical discoveries from program search with large language models (2023, background): “To surpass the ‘nominal’ capabilities of LLMs, recent studies 3 have combined them with evolutionary algorithms 14,15 , leading to important improvements on diverse synthetic problems 16 , searching for neural network architectures 17–19 and solving puzzles 20 .”
Learning Performance-Improving Code Edits (2023, methodology): “This methodology aligns with self-play and self-instruct approachs in neural program synthesis (Haluptzok et al., 2022; Rozière et al., 2023).”