A Conversational Paradigm for Program Synthesis

By Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Haiquan Wang, Yingbo Zhou, Silvio Savarese, Caiming Xiong. 2022

194 citations recorded by the source corpus.

Subfields: Information Systems, Software, Natural Language Processing, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

1. Uri Alon
2. Xi Victoria Lin
3. Graham Neubig
4. Baishakhi Ray
5. Ge Li
6. Zhi Jin
7. Oleksandr Polozov
8. Luke S. Zettlemoyer

In Software

1. Xi Victoria Lin
2. Oleksandr Polozov
3. Baishakhi Ray
4. Charles Sutton
5. Ge Li
6. Zhi Jin
7. Nan Duan
8. Luke S. Zettlemoyer

In Natural Language Processing

1. Graham Neubig
2. Luke S. Zettlemoyer
3. Mohit Bansal
4. Percy Liang
5. Caiming Xiong
6. Jianfeng Gao
7. Yejin Choi
8. Jason Aaron Edward Weston

In Code Generation & Program Synthesis

1. Graham Neubig
2. Paul Denny
3. Charles Sutton
4. Uri Alon
5. Wasi Uddin Ahmad
6. Xi Victoria Lin
7. Shuyan Zhou
8. Ge Li

Citations in the corpus

• OPT: Open Pre-trained Transformer Language Models (2022, methodology): “2Exceptions include work by EleutherAI, who released dense models up to 20B in size (Black et al., 2022), Salesforce (Nijkamp et al., 2022), and Meta AI, who released dense models up to 13B and sparse models up to 1.1T (Artetxe et al., 2021).”
• InCoder: A Generative Model for Code Infilling and Synthesis (2022, methodology): “While differences in details of the Transformer model architectures, datasets, and training procedures across papers and experimental setups make a rigorous comparison impossible, we note that our model achieves roughly comparable performance on the HumanEval metrics to CodeGen-Multi [46], which is ”
• CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X (2023, methodology): “Our contributions can be summarized as follows: • We develop and release CodeGeeX, a 13B pre-trained 23-language code generation model that demonstrates consistent outperfor-mance on code generation and translation over its multilingual baselines of the same scale.”
• Execution-Based Evaluation for Open-Domain Code Generation (2022, methodology): “, 2021), MTPB (Nijkamp et al., 2022), P3 (Haluptzok et al.”
• The Stack: 3 TB of permissively licensed source code (2022, methodology): “Several groups have explored decoder-only models with a causal language modeling objective (Chen et al., 2021; Austin et al., 2021; Nijkamp et al., 2022; Christopoulou et al., 2022; Izadi et al., 2022; Xu et al., 2022) and generally found that larger models are increasingly capable of synthesizing p”
• DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation (2022, background): “In contrast, other models like CodeGen-6B or InCoder-6B have much worse overall performance, with accuracy lower than 5% on some libraries.”
• Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants (2022, background): “Examples include OpenAI’s Codex [3, 5], AI21’s Jurrasic J1 [13,14], Salesforce CodeGen [15] and CodeBERT [16].”
• CodeT: Code Generation with Generated Tests (2022, methodology): “, 2022a), CODEGEN (Nijkamp et al., 2022) and PolyCoder Xu et al.”
• Multi-lingual Evaluation of Code Generation Models (2022, methodology): “The CodeGen-multi refers to the models at the end of the code pretraining stage without the python-only training, while the CodeGen-mono is models at the end of all three training stages.”
• Code4Struct: Code Generation for Few-Shot Structured Prediction from Natural Language (2022, methodology): “Such Code-LLMs have the ability to convert text into corresponding code as demonstrated by (Chen et al., 2021; Nijkamp et al., 2022).”
• LongCoder: A Long-Range Pre-trained Language Model for Code Completion (2023, background): “Codex (Chen et al., 2021), PolyCoder (Xu et al., 2022), CodeGen (Nijkamp et al., 2022), InCoder (Fried et al., 2022), and AlphaCode (Li et al., 2022) build large language models with billions of parameters and achieve impressive performance on code generation by training on a large-scale and high-qu”
• Parsel: A Unified Natural Language Framework for Algorithmic Reasoning (2022, background): “One alternative that has been explored is conversational code generation (Nijkamp et al., 2022; Yin et al., 2022).”
• ReCode: Robustness Evaluation of Code Generation Models (2022, methodology): “, 2022) and CodeGen-16B-mono (Nijkamp et al., 2022) where they perform correctly on regular prompts but fail on our perturbed ones after docstring paraphrasing and function camel case renaming in our ReCode benchmark.”
• Voyager: An Open-Ended Embodied Agent with Large Language Models (2023, background): “[81] Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong.”