Prompting GPT-3 To Be Reliable

By Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber, Lijuan Wang. International Conference on Learning Representations, 2022

378 citations recorded by the source corpus.

Large language models (LLMs) show impressive abilities via few-shot prompting. Commercialized APIs such as OpenAI GPT-3 further increase their use in real-world language applications. However, the crucial problem of how to improve the reliability of GPT-3 is still under-explored. While reliability is a broad and vaguely defined term, we decompose reliability into four main facets that correspond to the existing framework of ML safety and are well-recognized to be important: generalizability, social biases, calibration, and factuality. Our core contribution is to establish simple and effective prompts that improve GPT-3's reliability as it: 1) generalizes out-of-distribution, 2) balances demographic distribution and uses natural language instructions to reduce social biases, 3) calibrates output probabilities, and 4) updates the LLM's factual knowledge and reasoning chains. With appropriate prompts, GPT-3 is more reliable than smaller-scale supervised models on all these facets. We release all processed datasets, evaluation scripts, and model predictions. Our systematic empirical study not only sheds new insights on the reliability of prompting LLMs, but more importantly, our prompting strategies can help practitioners more reliably use LLMs like GPT-3.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/c8d594f09413b1555970f43e68847c211235d60f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP (2022, methodology): “Si et al. (2022) achieve 25.2% EM with CoT prompting.”
A Survey on In-context Learning (2022, background): “Si et al. (2022) explored the possibility of editing LLMs’ memorized knowledge through in-context demonstrations, discovering that a larger model scale and a mix of demonstration examples improved ICL-based knowledge editing success rates.”
A Survey for In-context Learning (2023, background): “Si et al. (2022) made an initial trial on in-context knowledge updating.”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, methodology): “Prevalent prompt methodologies, such as ReAct and Self-Consistency, similarly trade-off time for enhanced performance.”
Specializing Smaller Language Models towards Multi-Step Reasoning (2023, background): “The problem here is how to generalize beyond the tuning data, as small models may simply overfit the tuning distribution but struggle to generalize when the distribution shifts (Liu et al., 2022; Si et al., 2022).”
REPLUG: Retrieval-Augmented Black-Box Language Models (2023, methodology): “Another line of retrieval-augmented LMs such as kNN-LM (Khandelwal et al., 2020; Zhong et al., 2022) retrieves a set of tokens and interpolates between the LM’s next token distribution and kNN distributions computed from the retrieved tokens at inference.”
Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models (2023, methodology): “Finally, many methods have been proposed to improve the quality of the generated responses in general, typically by better prompting (Zhou et al., 2023; Si et al., 2023; Wei et al., 2022).”
A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models (2024, neutral): “This innovative combination allows the model to condition its output generation on both the input query and latent documents provided by the DPR.”
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails (2023, background): “The most common approach to add user-defined programmable rails to an LLM is to use prompting, including prompt engineering and in-context learning (Brown et al., 2020), by prepending or ap-pending a specific text to the user input (Wang and Chang, 2022; Si et al., 2022).”
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, positive): “Existing research highlights that the behavior of LLMs can significantly vary based on the prompts given by users (Si et al., 2022; Zhu et al., 2023).”
Can We Edit Factual Knowledge by In-Context Learning? (2023, background): “Si et al. (2022) is the first to explore whether in-context learning can update knowledge in LLMs, and show that incorporating all kinds of demonstration increase the success rate of knowledge editing.”
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (2022, methodology): “Adding explicit mitigation instructions directly to the prompt can be an effective way to reduce biases (Si et al., 2022).”
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (2023, background): “Furthermore, entity substitution (Longpre et al., 2021; Chen et al., 2022; Si et al., 2023; Zhou et al., 2023) replaces all mentions of the answer entity in parametric memory with other entities to construct counter-memory.”
A Comprehensive Overview of Large Language Models (2023, neutral): “Aligning with Prompts: LLMs can be steered with prompts to generate desirable responses without training [156], [157].”