Unified Pre-training for Program Understanding and Generation

By Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang. 2021

938 citations recorded by the source corpus.

Code summarization and generation empower conversion between programming language (PL) and natural language (NL), while code translation avails the migration of legacy code from one PL to another. This paper introduces PLBART, a sequence-to-sequence model capable of performing a broad spectrum of program and language understanding and generation tasks. PLBART is pre-trained on an extensive collection of Java and Python functions and associated NL text via denoising autoencoding. Experiments on code summarization in the English language, code generation, and code translation in seven programming languages show that PLBART outperforms or rivals state-of-the-art models. Moreover, experiments on discriminative tasks, e.g., program repair, clone detection, and vulnerable code detection, demonstrate PLBART’s effectiveness in program understanding. Furthermore, analysis reveals that PLBART learns program syntax, style (e.g., identifier naming convention), logical flow (e.g., “if“ block inside an “else“ block is equivalent to “else if“ block) that are crucial to program semantics and thus excels even with limited annotations.

Subfields: Artificial Intelligence, Software, Natural Language Processing, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Luke S. Zettlemoyer
  2. Mohit Bansal
  3. Jianfeng Gao
  4. Graham Neubig
  5. Percy Liang
  6. Caiming Xiong
  7. Yejin Choi
  8. Hannaneh Hajishirzi

In Software

  1. Chun Xia
  2. Lingming Zhang
  3. Prem Devanbu
  4. Yinlin Deng
  5. Toufique Ahmed
  6. Haoran Peng
  7. Chenyuan Yang
  8. Gabriel Synnaeve

In Natural Language Processing

  1. Graham Neubig
  2. Luke S. Zettlemoyer
  3. Mohit Bansal
  4. Mirella Lapata
  5. Jianfeng Gao
  6. Percy Liang
  7. Wen-tau Yih
  8. Dan Roth

In Code Generation & Program Synthesis

  1. Prem Devanbu
  2. Frank F. Xu
  3. Graham Neubig
  4. Chun Xia
  5. Lingming Zhang
  6. Toufique Ahmed
  7. Vincent J. Hellendoorn
  8. Yangruibo Ding

Citations in the corpus