CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks

By Ruchi Puri, David S. Kung, G. Janssen, Wei Zhang, Giacomo Domeniconi, Vladmir A. Zolotov, Julian Dolby, Jie Chen, Mihir R. Choudhury, Lindsey Decker and 4 more. 2021

391 citations recorded by the source corpus.

Over the last several decades, software has been woven into the fabric of every aspect of our society. As software development surges and code infrastructure of enterprise applications ages, it is now more critical than ever to increase software development productivity and modernize legacy applications. Advances in deep learning and machine learning algorithms have enabled numerous breakthroughs, motivating researchers to leverage AI techniques to improve software development efficiency. Thus, the fast-emerging research area of AI for Code has garnered new interest and gathered momentum. In this paper, we present a large-scale dataset CodeNet, consisting of over 14 million code samples and about 500 million lines of code in 55 different programming languages, which is aimed at teaching AI to code. In addition to its large scale, CodeNet has a rich set of high-quality annotations to benchmark and help accelerate research in AI techniques for a variety of critical coding tasks, including code similarity and classification, code translation between a large variety of programming languages, and code performance (runtime and memory) improvement techniques. Additionally, CodeNet provides sample input and output test sets for 98.5% of the code samples, which can be used as an oracle for determining code correctness and potentially guide reinforcement learning for code quality improvements. As a usability feature, we provide several pre-processing tools in CodeNet to transform source code into representations that can be readily used as inputs into machine learning models. Results of code classification and code similarity experiments using the CodeNet dataset are provided as a reference. We hope that the scale, diversity and rich, high-quality annotations of CodeNet will offer unprecedented research opportunities at the intersection of AI and Software Engineering.

Subfields: Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

In Natural Language Processing

Citations in the corpus

Competition-level code generation with AlphaCode (2022, methodology): “More concretely, the training dataset combines newly scraped data from Codeforces (Mirzayanov, 2020) with existing data from Description2Code (Caballero et al., 2016), and CodeNet (Puri et al., 2021).”
Self-Refine: Iterative Refinement with Self-Feedback (2023, methodology): “…Generation (Appendix M; Mehri and Eskenazi, 2020), Code Optimization (Appendix N; Madaan et al., 2023), Code Readability Improvement (Appendix L; Puri et al., 2021), Math Reasoning (Appendix O; Cobbe et al., 2021), Sentiment Reversal (Appendix P; Zhang et al., 2015), and we introduce two new…”
The Stack: 3 TB of permissively licensed source code (2022, background): “CodeNet (Puri et al., 2021) is another large code dataset collected from online programming contest websites.”
xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval (2023, background): “The problem solving nature and scale of X C ODE E VAL makes it ideal for other related applications in addition to single-task training and evaluation, such as prompting, instructional-tuning and multi-task learning (Sanh et al., 2021; Wei et al., 2021; Muennighoff et al., 2022).”
Learning Performance-Improving Code Edits (2023, methodology): “We also calculated the upper limit for speedup for the test set by aggregating the fastest submission across all submissions from CodeNet (118,841 accepted solutions) versus the fastest generation from our fastest model when increasing samples to 40 in Section 3.2.”
Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study (2026, unknown): “For C/C++ tasks (clone retrieval on POJ-104 and CodeForces title, tag matching, and tag classification), single-source pre-training relies exclusively on CodeNet, which provides samples from a single benchmark-style corpus.”
Lost in Translation, Found in Summary: Summary-Driven Supervision for Cross-Language Code Retrieval (2026, unknown): “…i.e., the number of cross-language problems (previous works) or groups (ours), significantly exceeds that of existing widely used cross-language datasets, including Co-deNet [40, 68], the training dataset used by previous methods (e.g., UniCoR [68]), and other test datasets described below.”
CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval (2025, unknown): “For correctness and efficiency , we select problems from CodeNet [10] that are not in CoQuIR, focusing on those with submissions in at least four different programming languages and containing both positive and negative examples, as defined in CoQuIR.”