Improving Automatic Source Code Summarization via Deep Reinforcement Learning
By Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, Philip S. Yu. 2018
433 citations recorded by the source corpus.
Code summarization provides a high level natural language description of the function performed by code, as it can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, most state-of-the-art approaches follow an encoder-decoder framework which encodes the code into a hidden space and then decode it into natural language space, suffering from two major drawbacks: a) Their encoders only consider the sequential content of code, ignoring the tree structure which is also critical for the task of code summarization; b) Their decoders are typically trained to predict the next word by maximizing the likelihood of next ground-truth word with previous ground-truth word given. However, it is expected to generate the entire sequence from scratch at test time. This discrepancy can cause an exposure bias issue, making the learnt decoder suboptimal. In this paper, we incorporate an abstract syntax tree structure as well as sequential content of code snippets into a deep reinforcement learning framework (i.e., actor-critic network). The actor network provides the confidence of predicting the next word according to current state. On the other hand, the critic network evaluates the reward value of all possible extensions of the current state and can provide global guidance for explorations. We employ an advantage reward composed of BLEU metric to train both networks. Comprehensive experiments on a real-world dataset show the effectiveness of our proposed model when compared with some state-of-the-art methods.
Subfields: Artificial Intelligence, Information Systems, Reinforcement Learning, Code Generation & Program Synthesis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Zhiyuan Liu
- Maosong Sun
- Nan Duan
- Prem Devanbu
- Mohit Bansal
- Jianfeng Gao
- Graham Neubig
- Toufique Ahmed
In Information Systems
- Prem Devanbu
- Toufique Ahmed
- Baishakhi Ray
- Nan Duan
- Miltiadis Allamanis
- Daya Guo
- Saikat Chakraborty
- Ge Li
In Reinforcement Learning
- Huan Sun
- Dongmei Zhang
- Zhengran Zeng
- Zhi Jin
- Guandong Xu
- Maosong Sun
- Patrick Fernandes
- Marc Brockschmidt
In Code Generation & Program Synthesis
- Prem Devanbu
- Toufique Ahmed
- Baishakhi Ray
- Wasi Uddin Ahmad
- Hongyu Zhang
- Ge Li
- Miltiadis Allamanis
- Saikat Chakraborty
Citations in the corpus
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, background): “• code-code (clone detection [65, 84, 46, 80, 9, 89, 86], defect detection [91, 55, 51, 42, 78, 79], cloze test [15], code completion [57, 1, 56, 66, 67, 8, 27, 7], code repair [69, 4, 23, 26, 71, 73], and code-to-code translation [48, 37, 10, 41]) • text-code (natural language code search [32, 20, ”
- Large Language Models for Software Engineering: A Systematic Literature Review (2023, background): “The applicability of LLMs is particularly pronounced in tasks such as code summarization [274], which involves yielding an abstract natural language depiction of a code’s functionality, as well as the generation of well-structured code [316] and code artifacts like annotations [162].”
- ChatDev: Communicative Agents for Software Development (2023, background): “…cycle, to segment the process into discrete phases ( e.g., design, coding, testing) and apply deep learning to improve the effectiveness of certain phases (Winkler et al., 2020; Ezzini et al., 2022; Thaller et al., 2019; Zhao et al., 2021; Nijkamp et al., 2023; Wan et al., 2018; Wang et al., 2021).”
- A Transformer-based Approach for Source Code Summarization (2020, methodology): “Among other noteworthy works, API usage information (Hu et al., 2018b), reinforcement learning (Wan et al., 2018), dual learning (Wei et al., 2019), retrieval-based techniques (Zhang et al., 2020) are leveraged to further enhance the code summarization models.”
- CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning (2019, methodology): “hey aim to generate annotations similar to the human-provided ones, and therefore trained the models using the standard maximum likelihood estimation (MLE) objective. For the same purpose, Wan et al. [58] trained the code annotation model in a reinforcement learning (RL) framework with reward being ”
- Multilingual training for Software Engineering (2021, background): “Comment synthesis is now an active research area, including many projects such as CodeNN [30], DeepCom [26], Astattgru [40], CodeBERT [18], Rencos [74], SecNN [42], PLBART [1], CoTexT [54], ProphetNet-X [55], NCS [2], Code2seq [7], Re2Com [71], and many more [19, 24, 25, 27, 28, 38, 39, 41, 49, 50, ”
- Retrieval-based Neural Source Code Summarization (2020, methodology): “For Hybrid-DRL, it has better performance than TL-CodeSum on PCSD since the AST-based structural information of source code is incorporated and the exposure bias problem is solved.”
- DLFix: Context-based Code Transformation Learning for Automated Program Repair (2020, methodology): “We adopted a DL-based code summarization model [41] to summarize a tree into a vector for a node (called a summarized node).”
- TranS^3: A Transformer-based Framework for Unifying Code Summarization and Code Search (2020, methodology): “For Hybrid-DeepCom, AutoSum, and TranS3 which all utilize the generated comments to strengthen their code search performance, their adopted code summarization metrics are essentially based on word frequency which generally are ne-grained, e.g., BLEU-based metrics, while their code search metrics are”
- Improved Code Summarization via a Graph Neural Network (2020, neutral): “[56] x x 2019 LeClair et al.”
- The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development (2023, methodology): “Codefluent large language models are capable of generating code from natural language descriptions [105], translating code from one language to another [75], generating unit tests [92], and even generating documentation for code [36, 38, 97, 98].”
- Communicative Agents for Software Development (2023, background): “(4) Software testing is to verify that the software can provide expected behaviors on a set of test cases [42; 40; 43; 39].”
- CoCoGUM: Contextual Code Summarization with Multi-Relational GNN on UMLs (2020, methodology): “Suppose 𝐴 and 𝐵 are generated and reference summaries of lengths c and r respectively, we have: 𝑅𝑂𝑈𝐺𝐸 − 𝐿 , which indicates the value of ROUGE-L, is calculated as the weighted harmonic mean of 𝑃 𝑅𝑂𝑈𝐺𝐸 − 𝐿 and 𝑅 𝑅𝑂𝑈𝐺𝐸 − 𝐿 : 𝛽 is set to 1 .”
- Structured Neural Summarization (2018, background): “Wan et al. (2018) instead directly operate on the tree structure using tree recurrent neural networks (Tai et al., 2015).”
- Multi-task Learning based Pre-trained Language Model for Code Completion (2020, methodology): “stical properties as natural language. Thus, statistical language models have been used for source code modeling [19, 42, 46], benefiting many software engineering tasks, including code summarization [23, 49], code clone detection [51, 52] program repair [15, 47], especially, in code completion [17,”