CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation

By Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang and 13 more. 2021

1,545 citations recorded by the source corpus.

Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT-style, GPT-style, and Encoder-Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.

Subfields: Information Systems, Software, Natural Language Processing, Evaluation & Benchmarks, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

  1. Chun Xia
  2. Lingming Zhang
  3. Uri Alon
  4. Graham Neubig
  5. Prem Devanbu
  6. Charles Sutton
  7. Miltiadis Allamanis
  8. Vincent J. Hellendoorn

In Software

  1. Chun Xia
  2. Lingming Zhang
  3. Prem Devanbu
  4. Toufique Ahmed
  5. Vincent J. Hellendoorn
  6. Baishakhi Ray
  7. Charles Sutton
  8. Baptiste Rozière

In Natural Language Processing

  1. Graham Neubig
  2. Luke S. Zettlemoyer
  3. Mohit Bansal
  4. Jianfeng Gao
  5. Caiming Xiong
  6. Percy Liang
  7. Richard Socher
  8. Kyunghyun Cho

In Evaluation & Benchmarks

  1. Luke S. Zettlemoyer
  2. Dan Roth
  3. Caiming Xiong
  4. Graham Neubig
  5. Quoc V. Le
  6. Thomas Scialom
  7. Michael Lewis
  8. Wen-tau Yih

In Signal Processing

  1. Gabriel Synnaeve
  2. Yossi Adi
  3. Quoc V. Le
  4. Graham Neubig
  5. Tal Remez
  6. Jade Copet
  7. Charles Sutton
  8. Lingming Zhang

Citations in the corpus