PromptChainer: Chaining Large Language Model Prompts through Visual Programming
By Tongshuang Sherry Wu, Ellen Jiang, Aaron Donsbach, J. Gray, A. Molina, Michael Terry, Carrie J. Cai. 2022
301 citations recorded by the source corpus.
While LLMs have made it possible to rapidly prototype new ML functionalities, many real-world applications involve complex tasks that cannot be easily handled via a single run of an LLM. Recent work has found that chaining multiple LLM runs together (with the output of one step being the input to the next) can help users accomplish these more complex tasks, and in a way that is perceived to be more transparent and controllable. However, it remains unknown what users need when authoring their own LLM chains – a key step to lowering the barriers for non-AI-experts to prototype AI-infused applications. In this work, we explore the LLM chain authoring process. We find from pilot studies that users need support transforming data between steps of a chain, as well as debugging the chain at multiple granularities. To address these needs, we designed PromptChainer, an interactive interface for visually programming chains. Through case studies with four designers and developers, we show that PromptChainer supports building prototypes for a range of applications, and conclude with open questions on scaling chains to even more complex tasks, as well as supporting low-fi chain prototyping.
Subfields: Artificial Intelligence, Health Informatics, Safety Research, Prompting & Reasoning, LLM Agents & Tool Use
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Percy Liang
- Luke S. Zettlemoyer
- Yejin Choi
- Jianfeng Gao
- Chitta Baral
- Xipeng Qiu
- Hannaneh Hajishirzi
In Health Informatics
In Safety Research
In Prompting & Reasoning
- Swaroop Mishra
- Chitta Baral
- Mihir Parmar
- Percy Liang
- Man Luo
- Qian Yang
- J.D. Zamfirescu-Pereira
- Bjoern Hartmann
In LLM Agents & Tool Use
- Swaroop Mishra
- Mihir Parmar
- Chitta Baral
- Man Luo
- Percy Liang
- Qian Yang
- J.D. Zamfirescu-Pereira
- Bjoern Hartmann
Citations in the corpus
- Augmented Language Models: a Survey (2023, methodology): “…al., 2019), or solve the sub-problems sequentially, where the solution to the next sub-problem depends on the answer to the previous ones (Yang et al., 2022a; Zhou et al., 2022; Drozdov et al., 2022; Dua et al., 2022; Khot et al., 2022; Wang et al., 2022a; Wu et al., 2022b; Mishra and Nouri, 2022).”
- AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts (2021, background): “Though beyond the scope of this paper, this raises the question of how to support users in not just using Chains, but also authoring their own Chains, to improve user agency and intuitiveness of Chaining [69].”
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models (2023, unknown): “texts, enabling more powerful reasoning [21, 47, 72, 23, 50, 71, 72].”
- Generative Agents: Interactive Simulacra of Human Behavior (2023, background): “…environment like the one we inhabit [10, 59], and strive to behave in ways that exhibit emergent behaviors grounded in social interactions with users or other agents with the aim of becoming believable proxies of our behavior in hypothetical simulations of individuals and communities [20, 36, 71].”
- Social Simulacra: Creating Populated Prototypes for Social Computing Systems (2022, methodology): “To generate these behaviors appropriately and reliably, we introduce prompt chains [80, 81] using GPT-3, a large language model [6], that (1) generate a large number of member personas based on a set of seed personas provided by the designer, then (2) generate from this large set of personas a set o”
- Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (2023, methodology): “In this work, we use these previously-identifed challenges to better understand why we observe some of the struggles with end-user prompt engineering that we do.”
- Character-LLM: A Trainable Agent for Role-Playing (2023, background): “Many attempts utilize prompted LLM (Wu et al., 2022a,b) that generate short natural language descriptions of per-sonas and the according behaviors, then they use the generated information to simulate social behaviors with language models (Park et al., 2022; Kim et al., 2022).”
- Large Language Models: A Survey (2024, unknown): “In “PromptChainer: Chaining Large Language Model Prompts through Visual Programming” [162], the authors not only describe the main challenges in designing chains, but also describe a visual tool to support those tasks.”
- BioTABQA: Instruction Learning for Biomedical Table Question Answering (2022, methodology): “approaches have been proposed recently using instructions to improve model performance [32, 33, 34, 35, 36].”
- GPT-4 passes the bar exam (2024, background): “While many real tasks in industry and society broadly require more knowledge and ability than tested on the Exam itself, there is significant opportunity to advance the performance of large language models through external queries, scratchpads, chain-of-thought prompting (CoT), or one of the many ot”
- Intelligent Clinical Documentation: Harnessing Generative AI for Patient-Centric Clinical Note Generation (2024, methodology): “ Prompt Chaining : [14] We experimented with breaking down the note generation task into smaller subtasks and chaining the prompts together, allowing for a more structured and controlled generation process.”
- In-BoXBART: Get Instructions into Biomedical Multi-Task Learning (2022, methodology): “Weller et al. (2020) and Ye and Ren (2021a) for improving task instructions for large LMs. Recently, many approaches have been proposed to improve model performance using instructions (Wu et al., 2021, 2022; Lin et al., 2021; Kuznia et al., 2022).”