Competition-level code generation with AlphaCode

By Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom, Eccles, James Keeling, Felix Gimeno and 22 more. 2022

2,356 citations recorded by the source corpus.

Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist programmers or even generate programs themselves could make programming more productive and accessible. Recent transformer-based neural network models show impressive code generation abilities yet still perform poorly on more complex tasks requiring problem-solving skills, such as competitive programming problems. Here, we introduce AlphaCode, a system for code generation that achieved an average ranking in the top 54.3% in simulated evaluations on recent programming competitions on the Codeforces platform. AlphaCode solves problems by generating millions of diverse programs using specially trained transformer-based networks and then filtering and clustering those programs to a maximum of just 10 submissions. This result marks the first time an artificial intelligence system has performed competitively in programming competitions. Description Machine learning systems can program too Computer programming competitions are popular tests among programmers that require critical thinking informed by experience and creating solutions to unforeseen problems, both of which are key aspects of human intelligence but challenging to mimic by machine learning models. Using self-supervised learning and an encoder-decoder transformer architecture, Li et al. developed AlphaCode, a deep-learning model that can achieve approximately human-level performance on the Codeforces platform, which regularly hosts these competitions and attracts numerous participants worldwide (see the Perspective by Kolter). The development of such coding platforms could have a huge impact on programmers’ productivity. It may even change the culture of programming by shifting human work to formulating problems, with machine learning being the main one responsible for generating and executing codes. —YS Modern machine learning systems can achieve average human-level performance in popular competitive programming contests.

Subfields: Artificial Intelligence, Information Systems, Software, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Luke S. Zettlemoyer
  4. Percy Liang
  5. Quoc V. Le
  6. Graham Neubig
  7. Yoshua Bengio
  8. Yejin Choi

In Information Systems

  1. Percy Liang
  2. Uri Alon
  3. Wen-tau Yih
  4. Xinyun Chen
  5. Luke S. Zettlemoyer
  6. Graham Neubig
  7. Denny Zhou
  8. Rishabh Singh

In Software

  1. Armando Solar-Lezama
  2. Rishabh Singh
  3. Oleksandr Polozov
  4. Josh Tenenbaum
  5. Percy Liang
  6. Pushmeet Kohli
  7. Koushik Sen
  8. Michihiro Yasunaga

In Code Generation & Program Synthesis

  1. Charles Sutton
  2. Paul Denny
  3. Armando Solar-Lezama
  4. Juho Leinonen
  5. Graham Neubig
  6. Sami Sarsa
  7. Arto Hellas
  8. Oleksandr Polozov

Citations in the corpus