Learning Execution through Neural Code Fusion
By Zhan Shi, Kevin Swersky, Daniel Tarlow, Parthasarathy Ranganathan, Milad Hashemi. 2019
32 citations recorded by the source corpus.
As the performance of computer systems stagnates due to the end of Moore's Law, there is a need for new models that can understand and optimize the execution of general purpose code. While there is a growing body of work on using Graph Neural Networks (GNNs) to learn representations of source code, these representations do not understand how code dynamically executes. In this work, we propose a new approach to use GNNs to learn fused representations of general source code and its execution. Our approach defines a multi-task GNN over low-level representations of source code and program state (i.e., assembly code and dynamic memory states), converting complex source code constructs and complex data structures into a simpler, more uniform format. We show that this leads to improved performance over similar methods that do not use execution and it opens the door to applying GNN models to new tasks that would not be feasible from static code alone. As an illustration of this, we apply the new model to challenging dynamic tasks (branch prediction and prefetching) from the SPEC CPU benchmark suite, outperforming the state-of-the-art by 26% and 45% respectively. Moreover, we use the learned fused graph embeddings to demonstrate transfer learning with high performance on an indirectly related task (algorithm classification).
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Charles Sutton
- David Bieber
- Daniel Tarlow
- Kevin Swersky
- Miltiadis Allamanis
- Parthasarathy Ranganathan
- Hugo Larochelle
- Danai Koutra
Citations in the corpus
- Neural Execution Engines: Learning to Execute Subroutines (2020, background): “Also, recent work has shown that graph neural networks are capable of learning from 64-bit binary memory states provided execution traces of assembly code, and observes that this representation numerically generalizes better than one-hot or categorical representations [21].”
- Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks (2021, methodology): “Degrees Layers [1, 2] [3, 7] [8, 21] [22, 60] [61, 168] 2 Acc 85.”
- Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks (2020, methodology): “Prior work applying machine learning for program analysis also operates under these restrictions [2, 13, 21, 22, 28, 29].”