A Transformer-based Approach for Source Code Summarization
By Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang. 2020
456 citations recorded by the source corpus.
Generating a readable summary that describes the functionality of a program is known as source code summarization. In this task, learning code representation by modeling the pairwise relationship between code tokens to capture their long-range dependencies is crucial. To learn code representation for summarization, we explore the Transformer model that uses a self-attention mechanism and has shown to be effective in capturing long-range dependencies. In this work, we show that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin. We perform extensive analysis and ablation studies that reveal several important findings, e.g., the absolute encoding of source code tokens’ position hinders, while relative encoding significantly improves the summarization performance. We have made our code publicly available to facilitate future research.
Subfields: Artificial Intelligence, Information Systems, Model Architectures, Code Generation & Program Synthesis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Graham Neubig
- Sergey Levine
- Yoshua Bengio
- Mohit Bansal
- Percy Liang
- Luke S. Zettlemoyer
- Pieter Abbeel
- Jianfeng Gao
In Information Systems
- Prem Devanbu
- Baishakhi Ray
- Toufique Ahmed
- Graham Neubig
- Saikat Chakraborty
- Yangruibo Ding
- Frank F. Xu
- Joseph E. Gonzalez
In Model Architectures
- Adam Santoro
- Baishakhi Ray
- Graham Neubig
- Prem Devanbu
- David Raposo
- Ajay Jain
- Pieter Abbeel
- Yangruibo Ding
In Code Generation & Program Synthesis
- Prem Devanbu
- Toufique Ahmed
- Frank F. Xu
- Graham Neubig
- Wasi Uddin Ahmad
- Yangruibo Ding
- Baishakhi Ray
- Shuyan Zhou
Citations in the corpus
- Unified Pre-training for Program Understanding and Generation (2021, background): “…Summarization (MovshovitzAttias and Cohen, 2013; Allamanis et al., 2016; Iyer et al., 2016; Alon et al., 2019a; Hu et al., 2018; Harer et al., 2019; Ahmad et al., 2020), Bug Detection (Ray et al., 2016; Li et al., 2018b; Russell et al., 2018; Zhou et al., 2019; Chakraborty et al., 2020), Program…”
- MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages (2022, background): “Code intelligence is a nascent but rapidly developing application of language processing, brewing an increasing number of code-relevant applications such as code summarization (Allamanis et al., 2016; Hu et al., 2018; Ahmad et al., 2020) and natural language (NL) to code generation (Ling et al.”
- HELoC: Hierarchical Contrastive Learning of Source Code Representation (2022, background): “…years, many deep learning based approaches to software engineering tasks, such as code retrieval [1, 2], code clone detection [3–6], code comment generation [7–9], software defect prediction [10–12], and code summarization [13–16], have emerged to improve software development and maintenance.”
- CoDesc: A Large Code–Description Parallel Dataset (2021, methodology): “…and English–French datasets (Bo-jar et al., 2014) containing 4.5 million and 36 million parallel sentences, respectively, whereas a similar network that achieved state-of-the-art results in source code summarization has been trained on only 69 thousand code-description pairs (Ahmad et al., 2020).”
- CODIT: Code Editing With Tree-Based Neural Models (2018, background): “Other NMT application in source code and software engineering include program comprehension [46], [49], [71], [72], commit message generation [73], [74], program synthesis [62], [75] etc.”
- Contrastive Code Representation Learning (2020, background): “Others explored summarization (Movshovitz-Attias and Cohen, 2013; Allamanis et al., 2016; Iyer et al., 2016; Ahmad et al., 2020) and types (Pradel et al.”
- Multilingual training for Software Engineering (2021, background): “Comment synthesis is now an active research area, including many projects such as CodeNN [30], DeepCom [26], Astattgru[40], CodeBERT [18], Rencos [74],SecNN [42],PLBART [1], CoTexT[54], ProphetNet-X [55], NCS [2], Code2seq[7], Re 2 Com [71], and many more [19, 24, 25, 27, 28, 38, 39, 41, 49, 50, 66,”
- Symbolic Behaviour in Artificial Intelligence (2021, background): “summaries of documents [34] or code [35].”