Neural Programmer: Inducing Latent Programs with Gradient Descent

By Arvind Neelakantan, Quoc V. Le, Ilya Sutskever. 2015

270 citations recorded by the source corpus.

Deep neural networks have achieved impressive supervised classification performance in many tasks including image recognition, speech recognition, and sequence to sequence learning. However, this success has not been translated to applications like question answering that may involve complex arithmetic and logic reasoning. A major limitation of these models is in their inability to learn even simple arithmetic and logic operations. For example, it has been shown that neural networks fail to learn to add two binary numbers reliably. In this work, we propose Neural Programmer, an end-to-end differentiable neural network augmented with a small set of basic arithmetic and logic operations. Neural Programmer can call these augmented operations over several steps, thereby inducing compositional programs that are more complex than the built-in operations. The model learns from a weak supervision signal which is the result of execution of the correct program, hence it does not require expensive annotation of the correct program itself. The decisions of what operations to call, and what data segments to apply to are inferred by Neural Programmer. Such decisions, during training, are done in a differentiable fashion so that the entire network can be trained jointly by gradient descent. We find that training the model is difficult, but it can be greatly improved by adding random noise to the gradient. On a fairly complex synthetic table-comprehension dataset, traditional recurrent networks and attentional models perform poorly while Neural Programmer typically obtains nearly perfect accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019, background): “Other re-lated work along these lines has been done by Reed and de Freitas (2015), Neelakantan et al. (2015), and Liang et al. (2017).”
A Survey of Machine Learning for Big Code and Naturalness (2017, background): “0] and Parisotto et al. [151] combine ideas from existing enumerative search techniques with learned heuristics to learn to eﬃciently synthesize code, usually written within a DSL. Neelakantan et al. [137] and Reed and de Freitas [167] introduce neural diﬀerentiable architectures for program inducti”
Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems (2017, background): “som, 2013), and inherits ideas from the extensive literature on semantic parsing (Jones et al., 2012; Berant et al., 2013; Andreas et al., 2013; Quirk et al., 2015; Liang et al., 2016; Neelakantan et al., 2016) and program generation (Reed and de Freitas, 2016; Graves et al.”
Programmable Agents (2017, background): “Different approaches to this problem focus on networks that infer programs [3, 10, 22], on networks that execute programs [39, 30, 5] and on networks that jointly do both [14, 17, 38, 29, 13]; although there is much overlap between many of these approaches.”
Supervising strong learners by amplifying weak experts (2018, background): “Algorithm learning: our problem differs from traditional work on learning algorithms (Graves et al., 2016; Kaiser and Sutskever, 2015; Neelakantan et al., 2015) because we don’t assume that we have access to ground truth labels.”
Neural Execution Engines: Learning to Execute Subroutines (2020, background): “Work in neural program synthesis [16, 18, 5, 1, 6]—which uses neural networks with the goal of generating and ﬁnding a “correct” program such that it will generalize beyond the training distribution—has also employed strong supervision in the form of execution traces [20, 21, 2].”
A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning (2019, methodology): “Later works on program induction (Reed and De Freitas, 2016; Neelakantan et al., 2016; Liang et al., 2017) extended this idea by using several built-in logic operations along with a key- value memory to learn different types of compositional programs such as addition or sorting.”
Neural Logic Machines (2019, methodology): “Neural program induction and synthesis (Neelakantan et al., 2016; Reed & De Freitas, 2016; Kaiser & Sutskever, 2016; Parisotto et al., 2017; Devlin et al., 2017; Bunel et al., 2018; Sun et al., 2018) are recently introduced to solve problems by synthesizing computer programs with neural…”
Making Neural Programming Architectures Generalize via Recursion (2017, background): “The training data for the Neural Programmer-Interpreter consists of full execution traces for the program of interest.”
Neural Module Networks (2015, background): “[31] describe a procedure for learning to assemble programs from a collection of functional primitives whose behavior is fully specified.”
Neural Architecture Search with Reinforcement Learning (2016, methodology): “…learning, probabilistic program induction has been used successfully in many settings, such as learning to solve simple Q&A (Liang et al., 2010; Neelakantan et al., 2015; Andreas et al., 2016), sort a list of numbers (Reed & de Freitas, 2015), and learning with very few examples (Lake et al.,…”
Neural Programmer-Interpreters (2015, neutral): “(Zaremba & Sutskever, 2015; Zaremba et al., 2015; Kaiser & Sutskever, 2015; Kurach et al., 2015; Neelakantan et al., 2015).”
RobustFill: Neural Program Learning under Noisy I/O (2017, methodology): “…Riedel et al., 2016), and (2) neural program induction , where network learns to generate the output directly using a latent program representation (Graves et al., 2014; 2016; Kurach et al., 2016; Kaiser & Sutskever, 2015; Joulin & Mikolov, 2015; Reed & de Fre-itas, 2016; Neelakantan et al., 2016).”
Inferring and Executing Programs for Visual Reasoning (2017, background): “Such models can take the form of a feedforward scoring function over operators in a domain-specific language that can be used to guide program search [4], or of a recurrent network that decodes a vectorial program representation into the actual program [22, 26, 34, 46, 47, 48].”
DeepCoder: Learning to Write Programs (2016, background): “Piech et al. (2015) propose to learn joint embeddings of program states and programs to automatically extend teacher feedback to many similar programs in the MOOC setting.”
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch (2020, neutral): “The synthesized programs—or algorithms—solve problems like sorting, addition, counting [74, 27, 70, 85], string manipulations [64, 61, 19], character recognition [38], competitionstyle programming [5], structured data QA [56, 45, 46],”
Continual and Multi-Task Architecture Search (2019, methodology): “NAS shares some similarity to program synthesis and inductive programming (Summers, 1986; Biermann, 1978), and it has been successfully applied to some simple Q&A tasks (Liang et al., 2010; Neelakantan et al., 2015; Andreas et al., 2016; Lake et al., 2015).”
Modular Multitask Reinforcement Learning with Policy Sketches (2016, background): “Another closely related family of models includes neural programmers (Neelakantan et al., 2015) and programmer–interpreters (Reed & de Fre-itas, 2016), which generate discrete computational structures but require supervision in the form of output actions or full execution traces.”