Leveraging Language to Learn Program Abstractions and Search Heuristics

By Catherine Wong, Kevin Ellis, Josh Tenenbaum, Jacob Andreas. 2021

61 citations recorded by the source corpus.

Inductive program synthesis, or inferring programs from examples of desired behavior, offers a general paradigm for building interpretable, robust, and generalizable machine learning systems. Effective program synthesis depends on two key ingredients: a strong library of functions from which to build programs, and an efficient search strategy for finding programs that solve a given task. We introduce LAPS (Language for Abstraction and Program Search), a technique for using natural language annotations to guide joint learning of libraries and neurally-guided search models for synthesis. When integrated into a state-of-the-art library learning system (DreamCoder), LAPS produces higher-quality libraries and improves search efficiency and generalization on three domains -- string editing, image composition, and abstract reasoning about scenes -- even when no natural language hints are available at test time.

Subfields: Artificial Intelligence, Natural Language Processing, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In LLM Agents & Tool Use

Citations in the corpus

Large Language Models Are Human-Level Prompt Engineers (2022, background): “Modern program synthesis admits a wide variety of specifications, including input-output examples (Ellis et al., 2021; Wong et al., 2021) and natural language (Jain et al.”
Skill Induction and Planning with Latent Language (2021, background): “…supervision and representation The use of natural language annotations to scaffold learning, especially in computer vision and program synthesis applications, has been the subject of a number of previous studies (Brana-van et al., 2009; Frome et al., 2013; Andreas et al., 2018; Wong et al., 2021).”
DreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning (2020, background): “[53], who also show how to incorporate a well-known human bias into the joint language–program inference problem [59].”
Vygotskian Autotelic Artificial Intelligence: Language and Culture Internalization for Human-Like AI (2022, background): “104 Generating plans to solve a particular drawing task helps shape the internal representation of the main policy which, they ﬁnd, favor abstraction and generalization in the main task.”
Learning to Synthesize Programs as Interpretable and Generalizable Policies (2021, background): “[56] Catherine Wong, Kevin Ellis, Joshua B Tenenbaum, and Jacob Andreas.”
CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive Program Synthesis (2025, unknown): “Finally, existing benchmarks for inductive program synthesis (Gulwani, 2011; Wang et al., 2017; Wong et al., 2021; Deng et al., 2024; Li & Ellis, 2025) are focused on domain-specific tasks and do not assess the ability of LLMs to synthesize functions written in general-purpose programming languages.”