A Novel Neural Source Code Representation Based on Abstract Syntax Tree
By Jian Zhang, Xu Wang, Hongyu Zhang, Hailong Sun, Kaixuan Wang, Xudong Liu. 2019
715 citations recorded by the source corpus.
Exploiting machine learning techniques for analyzing programs has attracted much attention. One key problem is how to represent code fragments well for follow-up analysis. Traditional information retrieval based methods often treat programs as natural language texts, which could miss important semantic information of source code. Recently, state-of-the-art studies demonstrate that abstract syntax tree (AST) based neural models can better represent source code. However, the sizes of ASTs are usually large and the existing models are prone to the long-term dependency problem. In this paper, we propose a novel AST-based Neural Network (ASTNN) for source code representation. Unlike existing models that work on entire ASTs, ASTNN splits each large AST into a sequence of small statement trees, and encodes the statement trees to vectors by capturing the lexical and syntactical knowledge of statements. Based on the sequence of statement vectors, a bidirectional RNN model is used to leverage the naturalness of statements and finally produce the vector representation of a code fragment. We have applied our neural network based source code representation method to two common program comprehension tasks: source code classification and code clone detection. Experimental results on the two tasks indicate that our model is superior to state-of-the-art approaches.
Subfields: Information Systems, Software, Code Generation & Program Synthesis, Program Analysis & Verification, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Information Systems
- Prem Devanbu
- Baishakhi Ray
- Daya Guo
- Nan Duan
- Ge Li
- Michael Pradel
- Saikat Chakraborty
- Georgios Gousios
In Software
- Prem Devanbu
- Neel Sundaresan
- Michael Pradel
- Daya Guo
- Vincent J. Hellendoorn
- Nan Duan
- Alexey Svyatkovskiy
- Georgios Gousios
In Code Generation & Program Synthesis
- Prem Devanbu
- Baishakhi Ray
- Ge Li
- Saikat Chakraborty
- Wasi Uddin Ahmad
- Zhi Jin
- Yunfei Zhao
- Yangruibo Ding
In Program Analysis & Verification
- Michael Pradel
- Prem Devanbu
- Georgios Gousios
- Petros Maniatis
- Ge Li
- Satish Chandra
- Vincent J. Hellendoorn
- Aditya Kanade
In Signal Processing
- Baishakhi Ray
- Saikat Chakraborty
- Ge Li
- Georgios Gousios
- Michael Pradel
- Prem Devanbu
- Vincent J. Hellendoorn
- Satish Chandra
Citations in the corpus
- Unified Pre-training for Program Understanding and Generation (2021, background): “, 2020), Clone Detection (Zhang et al., 2019; Yu et al., 2019; Wang et al., 2020), Code completion (Li et al.”
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, methodology): “ASTNN [89] uses RNNs to encode AST subtrees for statements.”
- TypeWriter: neural type prediction with search-based validation (2019, background): “ing-based techniques to find programming errors [25, 27], predict variable and method names [1, 28, 33], suggest how to improve names [19], search code [10, 30], detect clones [36, 40], classify code [24, 39], predict code edits [31, 38, 41], predict assertions [35], and automatically fix bugs [6, 1”
- CoCoGUM: Contextual Code Summarization with Multi-Relational GNN on UMLs (2020, methodology): “Previous works suggest various representations of source code for follow-up analysis [7, 12, 45–52].”
- Multi-task Learning based Pre-trained Language Model for Code Completion (2020, methodology): “nguage. Thus, statistical language models have been used for source code modeling [19, 42, 46], benefiting many software engineering tasks, including code summarization [23, 49], code clone detection [51, 52] program repair [15, 47], especially, in code completion [17, 19, 27, 46]. Code completion i”
- Structured Chain-of-Thought Prompting for Code Generation (2023, neutral): “Different from natural languages, source code contains rich structural information [22, 30, 37].”
- Learning and Evaluating Contextual Embedding of Source Code (2019, methodology): “…of variable and method identifiers using local and global context (Allamanis et al., 2015), abstract syntax trees or ASTs (Mou et al., 2016; Zhang et al., 2019), paths in ASTs (Alon et al., 2019), memory heap graphs (Li et al., 2016), and ASTs enriched with data-flow information…”
- RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024, methodology): “For each Python file, we apply Abstract Syntax Tree (AST) analysis (Zhang et al., 2019) to recursively parse the meta information of all Classes and Functions within the file, including their type, name, code snippets, etc.”
- MISIM: An End-to-End Neural Code Similarity System (2020, background): “In recent years, there has been an emergence of many MP systems, due, in part, to advances in machine learning, formal methods, data availability, and computing effi-ciency [1, 3–5, 10, 13, 29, 32, 41, 44].”
- Detecting Code Clones with Graph Neural Network and Flow-Augmented Abstract Syntax Tree (2020, methodology): “To leverage the explicit structural information in programs, these approaches often use abstract syntax tree (AST) as the input of their models [2]–[4].”
- HELoC: Hierarchical Contrastive Learning of Source Code Representation (2022, methodology): “Furthermore, TBCNN [25], ASTNN [24], and InferCode [4] (see Section 5.”
- Patching as Translation: the Data and the Metaphor (2020, background): “domain, either program languages (PL) or natural languages (NL). Existing work has especially spent effort on learning meaningful code representations to adapt seq2seq models for source code modeling [2, 3, 34]; these approaches focus on encoding rich characteristics of programming languages, beside”
- Retrieval-based Neural Source Code Summarization (2020, methodology): “Recently neural models are incorporated to learn vector representations of source code and use these vectors to compute code similarity [56, 57, 65, 67], which can achieve better performance but need additional time-consuming training and labeled data.”
- A Survey of Android Malware Detection with Deep Neural Models (2020, methodology): “• Tree-based representation: The tree structure can be employed to represent the Android apps, e.g., AST (Abstract Syntax Tree) is a tree representation of the abstract syntactic structure of the app’s source code [54].”
- Lost in Translation, Found in Summary: Summary-Driven Supervision for Cross-Language Code Retrieval (2026, unknown): “A natural idea is to rely on low-level IRs, e.g., LLVM IR [53], flow graphs [70], assembly code [21], and abstract syntax tree [72], and then perform matching or analysis in that unified space.”