Tree-to-tree Neural Networks for Program Translation

By Xinyun Chen, Chang Liu, Dawn Song. 2018

332 citations recorded by the source corpus.

Program translation is an important tool to migrate legacy code in one language into an ecosystem built in a different language. In this work, we are the first to employ deep neural networks toward tackling this problem. We observe that program translation is a modular procedure, in which a sub-tree of the source tree is translated into the corresponding target sub-tree at each step. To capture this intuition, we design a tree-to-tree neural network to translate a source tree into a target one. Meanwhile, we develop an attention mechanism for the tree-to-tree model, so that when the decoder expands one non-terminal in the target tree, the attention mechanism locates the corresponding sub-tree in the source tree to guide the expansion of the decoder. We evaluate the program translation capability of our tree-to-tree model against several state-of-the-art approaches. Compared against other neural translation models, we observe that our approach is consistently better than the baselines with a margin of up to 15 points. Further, our approach can improve the previous state-of-the-art program translation approaches by a margin of 20 points on the translation of real-world projects.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neural Architecture Search & AutoML, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Neural Architecture Search & AutoML

In Code Generation & Program Synthesis

Citations in the corpus

Unified Pre-training for Program Understanding and Generation (2021, background): “Code Translation requires a model to generate an equivalent code in the target PL from the input code written in the source PL. Note that the source and target PL can be the same.”
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis (2020, methodology): “Although there are several automatic evaluation methods, which can be used to evaluate code synthesis (Karaivanov, Raychev, and Vechev 2014; Chen, Liu, and Song 2018; Lachaux et al. 2020), these approaches still suffer from many weakness and are not suitable to evaluate code.”
Unsupervised Translation of Programming Languages (2020, methodology): “the percentage of translations that perfectly match the ground truth reference [12].”
Relational inductive biases, deep learning, and graph networks (2018, methodology): “…segment images and videos (Wang et al., 2018c; Hu et al., 2017) and 3D meshes and point clouds (Wang et al., 2018d), to classify regions in images (Chen et al., 2018a), to perform semi-supervised text classiﬁcation (Kipf and Welling, 2017), and in machine translation (Vaswani et al., 2017; Shaw…”
code2seq: Generating Sequences from Structured Representations of Code (2018, methodology): “There is a conceptual difference between our approaches: Hu et al. (2018) linearize the AST, and then pass it on to a standard seq2seq model.”
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages (2021, methodology): “Machine learning is frequently used on tasks involving programming languages, including code completion [28, 33, 35, 42], bug detection and code repair [15, 38, 46], code summarization [8, 25], clone detection [2, 47, 48], code search [13, 23] and code translation [14, 41].”
GraphCodeBERT: Pre-training Code Representations with Data Flow (2020, methodology): “Following Nguyen et al. (2015) and Chen et al. (2018), we conduct experiments on a dataset crawled from the same several open-source projects as them and report results in the Table 3.”
Multi-lingual Evaluation of Code Generation Models (2022, neutral): “Subsequent works used libraries and transcompilers to construct parallel corpora in Python 2 and Python 3 (Aggarwal et al., 2015), and CoffeeScript and JavaScript (Chen et al., 2018).”
Neural Guided Constraint Logic Programming for Program Synthesis (2018, background): “The use of graph or tree structure to represent programs [21, 22] and constraints [23] is not unprecedented.”
Structural Language Models for Any-Code Generation (2019, unknown): “Other work used datasets that are either small (Ling et al., 2016), containing highly aligned examples (Oda et al., 2015; Chen et al., 2018), limited-purpose languages like SQL (Yu et al., 2018), or general-purpose but containing eminently templated programs (Ling et al., 2016).”
Learning to Optimize Tensor Programs (2018, methodology): “A recent trend is to use deep neural networks to perform program analysis [3, 10].”
CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks (2021, background): “One considerable challenge of neural machine translation is that model training depends on large, parallel corpora that are expensive to curate [52], especially for low-resource languages (e.”
Code Translation with Compiler Representations (2022, background): “Indeed, apart from a few language pairs, such as Java-C# (Nguyen et al., 2013; Chen et al., 2018), and speciﬁc domains (e.g. competitive programming code), it is difﬁcult to collect large datasets of semantically equivalent code in different languages.”
A Survey on Large Language Models for Code Generation (2024, unknown): “A competent data mining expert needs a foundational education in computer science or related fields along with practical experience in programming languages (PLs), such as Python and R [2].”
Program Language Translation Using a Grammar-Driven Tree-to-Tree Model (2018, methodology): “After this, as in (Chen et al., 2018), we feed an embedded representation of tt into an LSTM to compute the hidden and cell state for each of its children.”
Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation (2020, methodology): “Grammar based methods have been used for a variety of tasks such as program translation [9], conditional program generation [71,72], grammar induction [32] and generative modelling on structures with syntax [37,12], such as molecules.”
PLUR: A Unifying, Graph-Based View of Program Learning, Understanding, and Repair (2021, methodology): “, 2018], or a more complex autoregressive structured decoder [Chen et al., 2018].”
Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem (2020, background): “…Alvarez-Melis and Jaakkola, 2017), grammar constrained decoders (Yin and Neubig, 2017; Yin et al., 2018; Jie and Lu, 2018; Dong and Lapata, 2018), action sequences for semantic graph generation (Chen et al., 2018a), and modular decoders based on abstract syntax trees (Rabinovich et al., 2017).”