AutoPandas: neural-backed generators for program synthesis

By Rohan Bavishi, Caroline Lemieux, Roy Fox, Koushik Sen, Ion Stoica. 2019

93 citations recorded by the source corpus.

Developers nowadays have to contend with a growing number of APIs. While in the long-term they are very useful to developers, many modern APIs have an incredibly steep learning curve, due to their hundreds of functions handling many arguments, obscure documentation, and frequently changing semantics. For APIs that perform data transformations, novices can often provide an I/O example demonstrating the desired transformation, but may be stuck on how to translate it to the API. A programming-by-example synthesis engine that takes such I/O examples and directly produces programs in the target API could help such novices. Such an engine presents unique challenges due to the breadth of real-world APIs, and the often-complex constraints over function arguments. We present a generator-based synthesis approach to contend with these problems. This approach uses a program candidate generator, which encodes basic constraints on the space of programs. We introduce neural-backed operators which can be seamlessly integrated into the program generator. To improve the efficiency of the search, we simply use these operators at non-deterministic decision points, instead of relying on domain-specific heuristics. We implement this technique for the Python pandas library in AutoPandas. AutoPandas supports 119 pandas dataframe transformation functions. We evaluate AutoPandas on 26 real-world benchmarks and find it solves 17 of them.

Subfields: Information Systems, Software, Natural Language Processing, Code Generation & Program Synthesis, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

In Natural Language Processing

In Code Generation & Program Synthesis

In Signal Processing

Citations in the corpus

InCoder: A Generative Model for Code Infilling and Synthesis (2022, background): “, input-output examples [12, 28, 19, 13], partial implementations [58], or natural language descriptions [74, 73, 72, 42, 18] of the desired program as input.”
Natural Language to Code Generation in Interactive Data Science Notebooks (2022, methodology): “Along this line of research, A UTO P ANDAS (Bavishi et al., 2019) synthesizes pandas programs for data transformation given examples of input/output DataFrame s.”
Jigsaw: Large Language Models meet Program Synthesis (2021, methodology): “So, out of 68 tasks in the PandasEval1 dataset and 21 tasks in the PandasEval2 dataset, only 20 and 7 are covered by the AutoPandas framework respectively.”
Gorilla: Large Language Model Connected with Massive APIs (2023, unknown): “Inductive program synthesis, where a program is synthesized to satisfy test cases, has found success in several avenues [3, 23].”
SpreadsheetCoder: Formula Prediction from Semi-structured Context (2021, background): “…various types of speciﬁcations have been discussed, including input-output examples (Gulwani et al., 2012; Balog et al., 2017; Bunel et al., 2018; Bavishi et al., 2019; Shin et al., 2018; Chen et al., 2019), natural language descriptions (Gulwani & Marron, 2014; Yu et al., 2018; Yin et al.,…”