Neural Programmer-Interpreters

By Scott E. Reed, Nando de Freitas. 2015

435 citations recorded by the source corpus.

We propose the neural programmer-interpreter (NPI): a recurrent and compositional neural network that learns to represent and execute programs. NPI has three learnable components: a task-agnostic recurrent core, a persistent key-value program memory, and domain-specific encoders that enable a single NPI to operate in multiple perceptually diverse environments with distinct affordances. By learning to compose lower-level programs to express higher-level programs, NPI reduces sample complexity and increases generalization ability compared to sequence-to-sequence LSTMs. The program memory allows efficient learning of additional tasks by building on existing programs. NPI can also harness the environment (e.g. a scratch pad with read-write pointers) to cache intermediate results of computation, lessening the long-term memory burden on recurrent hidden units. In this work we train the NPI with fully-supervised execution traces; each program has example sequences of calls to the immediate subprograms conditioned on the input. Rather than training on a huge number of relatively weak labels, NPI learns from a small number of rich examples. We demonstrate the capability of our model to learn several types of compositional programs: addition, sorting, and canonicalizing 3D models. Furthermore, a single NPI learns to execute these programs and all 21 associated subprograms.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019, background): “Other re-lated work along these lines has been done by Reed and de Freitas (2015), Neelakantan et al. (2015), and Liang et al. (2017).”
Adaptive Computation Time for Recurrent Neural Networks (2016, positive): “The evidence that increased depth leads to more performant networks is by now inarguable [5, 4, 19, 9], and recent results show that increased sequence length can be similarly beneficial [31, 33, 25].”
Show Your Work: Scratchpads for Intermediate Computation with Language Models (2021, neutral): “The related problem of neural algorithm induction has attracted considerable interest (Graves et al., 2014; Kurach et al., 2016; Kaiser & Sutskever, 2016; Graves et al., 2016; Reed & de Freitas, 2016; Veliˇckovi´c et al., 2020a;b).”
Evaluating Large Language Models Trained on Code (2021, methodology): “More recent approaches like the Neural Program Interpreter (Reed & de Freitas, 2016; Shin et al., 2018; Pierrot et al., 2021) and Universal Transformer (Dehghani et al., 2019) found recurrence to be a useful component in program induction.”
A Survey of Machine Learning for Big Code and Naturalness (2017, methodology): “alsosomeworkonprobabilistic modelsofcode that do not have a learning component, such as Liblit et al. [117]. Within machine learning, there hasbeenaninterestingrecentlineofworkonneuralabstractmachines[75,167,168],whichextend deterministic automatafromcomputerscience,suchaspushdownautomataandTuringma”
Programmable Agents (2017, background): “Compositionality has become a topic of great interest in machine learning, robotics and cognitive science [20, 30, 19, 9, 35, 26], and it has been central to research in language [32, 25, 33, 36].”
Neural Execution Engines: Learning to Execute Subroutines (2020, background): “Work in neural program synthesis [16, 18, 5, 1, 6]—which uses neural networks with the goal of generating and ﬁnding a “correct” program such that it will generalize beyond the training distribution—has also employed strong supervision in the form of execution traces [20, 21, 2].”
Neural Logic Machines (2019, methodology): “Neural program induction and synthesis (Neelakantan et al., 2016; Reed & De Freitas, 2016; Kaiser & Sutskever, 2016; Parisotto et al., 2017; Devlin et al., 2017; Bunel et al., 2018; Sun et al., 2018) are recently introduced to solve problems by synthesizing computer programs with neural…”
Making Neural Programming Architectures Generalize via Recursion (2017, methodology): “The training data for the Neural Programmer-Interpreter consists of full execution traces for the program of interest.”
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets (2022, background): “Some papers study the sample complexity on algorithmic tasks Reed & De Freitas (2015), but mostly focus on the impact of architectural choices.”
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, methodology): “The program synthesis 333 [47, 58, 62, 74, 17, 40, 90] extends the text-to-code generation task aims to generate programs from 334 a specification [21], such as pseudocode, natural language description, and input/output examples.”
Neural Architecture Search with Reinforcement Learning (2016, methodology): “…probabilistic program induction has been used successfully in many settings, such as learning to solve simple Q&A (Liang et al., 2010; Neelakantan et al., 2015; Andreas et al., 2016), sort a list of numbers (Reed & de Freitas, 2015), and learning with very few examples (Lake et al., 2015).”
RobustFill: Neural Program Learning under Noisy I/O (2017, background): “Neural Programmer-Interpreters (Reed & de Freitas, 2016) teach a controller to learn algorithms from program traces as opposed to examples.”
Neural Programmer: Inducing Latent Programs with Gradient Descent (2015, methodology): “After our work was submitted to ArXiv, Neural Programmer-Interpreters (Reed & Freitas, 2016), a method that learns to induce programs with supervision of the entire program was proposed.”
Inferring and Executing Programs for Visual Reasoning (2017, background): “The recurrent networks may incorporate compositional structure that allows them to learn new programs by combining previously learned sub-programs [35].”
Interaction Networks for Learning about Objects, Relations and Physics (2016, background): “19] is a framework that shares learning across nodes and edges, the “recursive autoencoder” [25] adapts its processing architecture to exploit an input parse tree, the “neural programmer-interpreter” [22] is a composable network that mimics the execution trace of a program, and the “spatial transfor”
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (2016, background): “In contrast, operations implemented in ordinary programming language are abstract , scalable , and precise , because no matter how large the input is or whether it has been seen or not, they will be processed precisely.”