Convolutional Neural Networks over Tree Structures for Programming Language Processing
By Lili Mou, Ge Li, Lu Zhang, Tao Wang, Zhi Jin. 2014
843 citations recorded by the source corpus.
Programming language processing (similar to natural language processing) is a hot research topic in the field of software engineering; it has also aroused growing interest in the artificial intelligence community. However, different from a natural language sentence, a program contains rich, explicit, and complicated structural information. Hence, traditional NLP models may be inappropriate for programs. In this paper, we propose a novel tree-based convolutional neural network (TBCNN) for programming language processing, in which a convolution kernel is designed over programs' abstract syntax trees to capture structural information. TBCNN is a generic architecture for programming language processing; our experiments show its effectiveness in two different program analysis tasks: classifying programs according to functionality, and detecting code snippets of certain patterns. TBCNN outperforms baseline methods, including several neural models for NLP.
Subfields: Artificial Intelligence, Information Systems, Software, Code Generation & Program Synthesis, Program Analysis & Verification
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Percy Liang
- Jianfeng Gao
- Luke S. Zettlemoyer
- Graham Neubig
- Yoshua Bengio
- Charles Sutton
- Quoc V. Le
In Information Systems
- Charles Sutton
- Michael Pradel
- Miltiadis Allamanis
- Koushik Sen
- Rishabh Singh
- Baishakhi Ray
- Ge Li
- Premkumar T. Devanbu
In Software
- Charles Sutton
- Michael Pradel
- Miltiadis Allamanis
- Rishabh Singh
- Koushik Sen
- Tien Nhut Nguyen
- Pushmeet Kohli
- Premkumar T. Devanbu
In Code Generation & Program Synthesis
- Charles Sutton
- Miltiadis Allamanis
- Michael Pradel
- Ge Li
- Rishabh Singh
- Hongyu Zhang
- Zhi Jin
- Baishakhi Ray
In Program Analysis & Verification
- Charles Sutton
- Michael Pradel
- Aditya Kanade
- Petros Maniatis
- Rishabh Singh
- Miltiadis Allamanis
- Tien Nhut Nguyen
- Georgios Gousios
Citations in the corpus
- Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “Mou et al. (2016) proposed a convolution operation on trees, while the locality are defined based on parent-child relations.”
- TypeWriter: neural type prediction with search-based validation (2019, background): “ing-based techniques to find programming errors [25, 27], predict variable and method names [1, 28, 33], suggest how to improve names [19], search code [10, 30], detect clones [36, 40], classify code [24, 39], predict code edits [31, 38, 41], predict assertions [35], and automatically fix bugs [6, 1”
- Natural Language Inference by Tree-Based Convolution and Heuristic Matching (2015, methodology): “The tree-based convolutoinal neural network (TBCNN) is first proposed in our previous work (Mou et al., 2016) 2 to classify program source code; later, we further propose TBCNN variants to model sentences (Mou et al., 2015).”
- Discriminative Neural Sentence Modeling by Tree-Based Convolution (2015, methodology): “The idea of tree-based convolution is to apply a set of subtree feature detectors, sliding over the entire 1The model of tree-based convolution was firstly proposed to process program source code in our (unpublished) previous work (Mou et al., 2014).”
- DeepBugs: a learning approach to name-based bug detection (2018, background): “; Godefroid et al. 2017; Liu et al. 2017]. Deep neural networks also help recommend API usages [Gu et al. 2016], detect code clones [White et al. 2016], classify programs into pre-defined categories [Mou et al. 2016], 18 and adapt copied-and-pasted code to its surrounding code [Allamanis and Brocksc”
- LambdaNet: Probabilistic Type Inference using Graph Neural Networks (2020, methodology): “In this paper, we propose a new probabilistic type inference algorithm for TypeScript to address these shortcomings using a graph neural network architecture (GNN) (Veliˇckovi´c et al., 2018; Li et al., 2016; Mou et al., 2016).”
- Learning Continuous Semantic Representations of Symbolic Expressions (2016, background): “Mou et al. (2016) design tree convolutional networks to classify code into student submission tasks.”
- Neural Program Repair by Jointly Learning to Localize and Repair (2019, background): “In particular, different representations based on token sequences (Gupta et al., 2017; Bhatia et al., 2018), program parse trees (Piech et al., 2015; Mou et al., 2016), program traces (Reed & de Freitas, 2015; Cai et al., 2017; Wang et al., 2018), and graphs (Allamanis et al., 2018) have been…”
- Variational Reasoning for Question Answering with Knowledge Graph (2017, background): “Graph embedding: Recently, researchers have built deep architectures to embed structured data, such as trees (Socher et al. 2013b; Irsoy and Cardie 2014; Mou et al. 2016) or graphs (Duvenaud et al.”
- Deep Code Comment Generation (2018, background): “[26] present a tree-based convolutional neural networks to model the source code and classify programs.”
- HELoC: Hierarchical Contrastive Learning of Source Code Representation (2022, methodology): “Furthermore, TBCNN [25], ASTNN [24], and InferCode [4] (see Section 5.”
- A Novel Neural Source Code Representation Based on Abstract Syntax Tree (2019, methodology): “Even though code fragments have something in common with plain texts, they should not be simply dealt with text-based or token-based methods due to their richer and more explicit structural information [2], [18].”
- Integrating Tree Path in Transformer for Code Representation (2021, background): “[Mou et al., 2016] proposed tree-based convolutional neural networks to represent source code.”
- Deep Code Search (2018, background): “Recently, researchers have investigated possible applications of deep learning techniques to source code [7, 38, 53, 56, 60, 64, 75, 76].”