A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
By Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, Douglas C. Schmidt. Proceedings of the 30th Conference on Pattern Languages of Programs, 2023
1,823 citations recorded by the source corpus.
Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.
Subfields: Artificial Intelligence, Computer Networks and Communications, Health Informatics
View Paper: https://www.semanticscholar.org/paper/08b85bce712168998004ee80ce4e475390413c74
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yejin Choi
- Cunxiang Wang
- Prateek Mittal
- Peter Henderson
- Alane Suhr
- Jindong Wang
- Xiangyu Qi
- Mohit Bansal
In Computer Networks and Communications
- Prateek Mittal
- Jindong Wang
- Yejin Choi
- Justin D. Weisz
- Melanie Sclar
- Matthew Renze
- Tianwei Zhang
- Erhan Guven
In Health Informatics
- Prateek Mittal
- Werner Geyer
- Siyuan Ma
- Michael J. Muller
- Yejin Choi
- Abdulhadi Shoufan
- Justin D. Weisz
- Gabriela Hoefer
Citations in the corpus
- Design Principles for Generative AI Applications (2024, background): “One emerging skill revolves around crafting efective natural language prompts, known as in-context learning [20, 40, 189] or prompt engineering [185, 193].”
- Security and Privacy Challenges of Large Language Models: A Survey (2024, neutral): “Our objective is to provide a clear vision for researchers, practitioners, and other stakeholders who plan to develop and/or deploy LLMs, in terms of the significance of LLM security and privacy challenges.”
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (2023, background): “As the capabilities of LLMs have rapidly improved, their sensitivity to input prompt features has been used to optimize performance via prompt engineering (White et al., 2023).”
- Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey (2023, background): “Prompt engineering is an emerging topic in large language models that provides an understanding of how prompting affects the performance of these models [32], [33].”
- The Effect of Sampling Temperature on Problem Solving in Large Language Models (2024, methodology): “• Self-recitation – the system prompt instructs the LLM to recite its own internal knowledge about the problem before answering the question [2,30].”
- A Survey on Evaluation of Large Language Models (2023, background): “One common approach to interacting with LLMs is prompt engineering (Clavié et al., 2023; White et al., 2023; Zhou et al., 2022), where users design and provide specific prompt texts to guide LLMs in generating desired responses or completing specific tasks.”
- Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models (2023, background): “[99] proposed a framework for cueing models applicable to different domains.”
- Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (2023, background): “[80] proposed a framework for cueing models applicable to different domains.”
- AI literacy and its implications for prompt engineering strategies (2024, background): “To accomplish effective communication with LLM based chatbots like ChatGPT, however, the ability to engineer effective prompts is becoming more and more important (White et al., 2023).”
- Multimodal Large Language Models: A Survey (2023, background): “It has the ability to handle zero or small data samples, which has been widely adopted in various LLMs [37].”
- Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study (2023, background): “Consequently, researchers have begun to employ prompt engineering [8] as a means of jailbreaking C HAT GPT. Prompt engineering involves selecting and fine-tuning prompts that are tailored to a specific task or application for which the LLM will be used.”
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (2023, result): “, 2022) and prompt engineering (White et al., 2023) do not require parameter changes, fine-tuning still remains preferable in many settings as it avoids additional inference-time overhead and often delivers better and more stable results (Hao et al.”
- A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models (2024, background): “Prompt engineering is the process of experimenting with various instructions to get the best output possible from an AI text generation model (White et al., 2023).”
- Large corpora and large language models: a replicable method for automating grammatical annotation (2024, unknown): “The fi rst step of the pipeline consists in prompt engineering, that is, the formulation of input prompts that help elicit the desired response from Claude (Anthropic n.d.; White et al. 2023).”