Automatic Prompt Optimization with "Gradient Descent" and Beam Search

By Reid Pryzant, Dan Iter, Jerry Zheng Li, Yin Tat Lee, Chenguang Zhu, Michael Zeng. Conference on Empirical Methods in Natural Language Processing, 2023

730 citations recorded by the source corpus.

Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. We propose a simple and nonparametric solution to this problem, Automatic Prompt Optimization (APO), which is inspired by numerical gradient descent to automatically improve prompts, assuming access to training data and an LLM API. The algorithm uses minibatches of data to form natural language"gradients"that criticize the current prompt. The gradients are then"propagated"into the prompt by editing the prompt in the opposite semantic direction of the gradient. These gradient descent steps are guided by a beam search and bandit selection procedure which significantly improves algorithmic efficiency. Preliminary results across three benchmark NLP tasks and the novel problem of LLM jailbreak detection suggest that Automatic Prompt Optimization can outperform prior prompt editing techniques and improve an initial prompt's performance by up to 31%, by using data to rewrite vague task descriptions into more precise annotation instructions.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/c76dd4a70361c3afd2e19d046343e2dedd16ecc3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Jailbreaking Black Box Large Language Models in Twenty Queries (2023, neutral): “There exist a variety of techniques for automatic prompting, [11, 24, 30].”
Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution (2023, methodology): “, 2022), APO (Pryzant et al., 2023), and APE (Zhou et al.”
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (2023, background): “Researchers are starting to apply discrete optimization and RL to find effective prompts, generally for a single logical LM call (Guo et al., 2023; Pryzant et al., 2023; Huang et al., 2022; Yang et al., 2023).”
Defending ChatGPT against jailbreak attack via self-reminders (2023, positive): “Then, our self-reminder optimization mechanism, which is based on an automatic prompt optimization 46 technique, uses these auto-generated prompts, along with a handcrafted one, as its initialization.”
Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models (2023, neutral): “For example, Pryzant et al. [36] propose using LLMs to provide suggestions for each prompt and optimize the prompt accordingly.”
Reflexion: language agents with verbal reinforcement learning (2023, background): “[19] performs a similar semantic prompt-writing optimization, but is also limited to single-generation tasks.”
ORPO: Monolithic Preference Optimization without Reference Model (2024, background): “However, despite the ability to follow instructions, models may generate harmful or unethical outputs (Carlini et al., 2021; Gehman et al., 2020; Pryzant et al., 2023).”
ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution (2024, positive): “…Shinn et al. (2023) leverage binary rewards indicating passing or failing test cases in programming tasks, ReEvo extends the scope of verbal RL feedback to comparative analysis of two heuristics, which can be analogous to verbal gradient information (Pryzant et al., 2023) within heuristic spaces.”
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (2023, methodology): “We use c = 2 following Pryzant et al. (2023), who find UCB with c = 2 to be most effective for prompt optimization.”
Large Language Models as Optimizers (2023, methodology): “Different from recent work on using LLMs for automatic prompt generation (Zhou et al., 2022b; Pryzant et al., 2023), each optimization step in our work generates new prompts that aim to increase the test accuracy based on a trajectory of previously generated prompts, instead of editing one input…”
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers (2024, neutral): “We reproduce APO on binary classification tasks with the optimal manual prompt as the initial one.”