Large Language Models as Optimizers

By Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, Xinyun Chen. International Conference on Learning Representations, 2023

977 citations recorded by the source corpus.

Optimization is ubiquitous. While derivative-based algorithms have been powerful tools for various problems, the absence of gradient imposes challenges on many real-world applications. In this work, we propose Optimization by PROmpting (OPRO), a simple and effective approach to leverage large language models (LLMs) as optimizers, where the optimization task is described in natural language. In each optimization step, the LLM generates new solutions from the prompt that contains previously generated solutions with their values, then the new solutions are evaluated and added to the prompt for the next optimization step. We first showcase OPRO on linear regression and traveling salesman problems, then move on to our main application in prompt optimization, where the goal is to find instructions that maximize the task accuracy. With a variety of LLMs, we demonstrate that the best prompts optimized by OPRO outperform human-designed prompts by up to 8% on GSM8K, and by up to 50% on Big-Bench Hard tasks. Code at https://github.com/google-deepmind/opro.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/f8a2dca1e8fe56e698984c077f7ff58d8ca867e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine (2023, background): “Concurrent with our study, recent works [35, 7] also find that foundation models write better prompts than experts do.”
Empowering Biomedical Discovery with AI Agents (2024, positive): “Additionally, the application of LLM as an optimizer to enhance prompt reﬁnement and optimization for improved performance in assigned roles has been investigated in [70].”
Evaluation and mitigation of the limitations of large language models in clinical decision-making (2024, neutral): “A subset of 80 representative patients of the MIMIC-CDM-FI data-set was randomly selected to be used for comparison with the physicians.”
Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities (2024, background): “[21] LLM generates new solutions from the prompt that contains previously generated solutions with their values, then the new solutions are evaluated and added to the prompt for the next optimization step.”
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (2023, background): “…a particular signature needs to specify parameters that include: (1) the specific LM to call (Chen et al., 2023), (2) the prompt instructions (Yang et al., 2023) and the string prefix of each signature field and, most importantly, (3) the demonstrations used as few-shot prompts (for frozen…”
Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model (2024, methodology): “Initial attempts have been made to couple LLMs with Evolutionary Computation (EC) to de-sign more effective algorithms (Yang et al., 2023; Meyerson et al., 2023; Chen et al., 2023).”
ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution (2024, neutral): “…(self-)improvement (Zelikman et al., 2023; Liu et al., 2023c;a), neural architecture search (Chen et al., 2023a), and general solution optimization (Yang et al., 2023a; Brooks et al., 2023; Wang et al., 2023a), with many under evolutionary frameworks (Wu et al., 2024; Hemberg et al., 2024; Chao et…”
Large Language Models Cannot Self-Correct Reasoning Yet (2023, background): “Additionally, such search efforts can be equally applied to the pre-hoc prompts (Zhou et al., 2022; Yang et al., 2023).”
A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications (2024, positive): “Active Prompting Diao et al. [2023] introduced Active-Prompting as a solution to the challenge of adapting LLMs to diverse reasoning tasks.”