Project CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks

By Ruchi Puri, David S. Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Giacomo Domeniconi, V. Zolotov, Vladimir Zolotov, Julian Dolby, Julian Dolby and 7 more. 2021

91 citations recorded by the source corpus.

Subfields: Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

1. Chandan K. Reddy
2. Jian Yin
3. Yanlin Wang
4. Shuai Lu
5. Daya Guo
6. Gabriel Synnaeve
7. Augustus Odena
8. Yanqiao Zhu

Citations in the corpus

• Show Your Work: Scratchpads for Intermediate Computation with Language Models (2021, neutral): “CodeNet The Project CodeNet dataset (Puri et al., 2021) consists of millions of user submissions to approximately 4,000 coding problems.”
• Program Synthesis with Large Language Models (2021, methodology): “Such datasets include the Natural Program Synthesis (NAPS) dataset [Zavershynskyi et al., 2018], the Search-based Pseudocode to Code (SPoC) dataset [Kulal et al., 2019], the APPS dataset [Hendrycks et al., 2021], the PROGRES dataset [Alet et al., 2021], and the CodeNet dataset [Puri et al., 2021].”
• A Survey on Pretrained Language Models for Neural Code Intelligence (2022, background): “Big-CloneBench (Svajlenko et al., 2014) and Co-deNet (Puri et al., 2021) are two representative benchmarks for evaluating the performance of code clone detection algorithms.”
• CodeMind: A Framework to Challenge Large Language Models for Code Reasoning (2024, methodology): “…al., 2021; Liu et al., 2023; Gu et al., 2024; Zheng et al., 2023; Cassano et al., 2022; Jimenez et al., 2023; Du et al., 2023; Odena et al., 2021; Puri et al., 2021; Ahmad et al., 2021), we chose the programs in HumanEval (Chen et al., 2021), MBPP (Odena et al., 2021), CodeNet (Puri et al.,…”
• Code Translation with Compiler Representations (2022, methodology): “To learn to decompile IRs, we also used the CodeNet dataset (Puri et al., 2021), a repository of 14 million competitive programming solutions in 55 languages.”
• XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence (2022, background): “Project CodeNet (Puri et al., 2021) has an abundance of parallel programs in a wide range of languages.”
• NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness (2024, background): “Simultaneously, the community has been actively designing benchmarks (Hendrycks et al., 2021; Chen et al., 2021; Austin et al., 2021; Puri et al., 2021; Li et al., 2022; Liu et al., 2023b) with emphasis on generating code for a given problem specification of what functionality to achieve, e.g.,…”
• CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation (2023, unknown): “Execution-Based Multilingual Multitask Multidimensional”
• UniXcoder: Unified Cross-Modal Pre-training for Code Representation (2022, methodology): “We collect 11,744/15,594/23,530 functions from the CodeNet corpus (Puri et al., 2021) in Ruby/Python/Java PL.”