Calibrate Before Use: Improving Few-Shot Performance of Language Models
By Tony Zhao, Eric Wallace, Shi Feng, Daniel Klein, Sameer Singh. 2021
1,964 citations recorded by the source corpus.
GPT-3 can perform numerous tasks when provided a natural language prompt that contains a few training examples. We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art. We demonstrate that this instability arises from the bias of language models towards predicting certain answers, e.g., those that are placed near the end of the prompt or are common in the pre-training data. To mitigate this, we first estimate the model's bias towards each answer by asking for its prediction when given the training prompt and a content-free test input such as "N/A". We then fit calibration parameters that cause the prediction for this input to be uniform across answers. On a diverse set of tasks, this contextual calibration procedure substantially improves GPT-3 and GPT-2's average accuracy (up to 30.0% absolute) and reduces variance across different choices of the prompt.
Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Jianfeng Gao
- Percy Liang
- Yejin Choi
- Graham Neubig
- Hannaneh Hajishirzi
- Quoc V. Le
In Large Language Models
- Luke S. Zettlemoyer
- Hinrich Schütze
- Yejin Choi
- Mohit Bansal
- Timo Schick
- Hannaneh Hajishirzi
- Jianfeng Gao
- Michael Lewis
In Prompting & Reasoning
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Denny Zhou
- Timo Schick
- Hinrich Schütze
- Swaroop Mishra
- Yejin Choi
- Michael Lewis
Citations in the corpus
- Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “The one exception is the coin flip task, for which exemplar orders have high standard deviation, likely for the reason cited in Zhao et al. (2021)—for classification, many exemplars of the same category in a row biases the model outputs).”
- Ethical and social risks of harm from Language Models (2021, methodology): “In response to a question, GPT-3 was found to frequently provide common, but false utterances, rather than providing the less common, correct utterance (Zhao et al., 2021).”
- Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, methodology): “…we consider ensembling by: (1) prompt order permutation: we randomly permute the exemplars in the prompt 40 times to mitigate model’s sensitivity to prompt order (Zhao et al., 2021; Lu et al., 2021); and (2) multiple sets of prompts (Gao et al., 2021): we manually write 3 different sets of prompts.”
- Evaluating Large Language Models Trained on Code (2021, background): “And research shows that so-called prompt engineering can enable stronger results from AI systems (Zhao et al., 2021).”
- Emergent Abilities of Large Language Models (2022, positive): “For instance, simple modifications such as calibrating output probabilities (Zhao et al., 2021; Holtzman et al., 2021) or using a noisy channel (Min et al., 2021) have improved performance on a range of tasks.”
- Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2021, methodology): “The results for different few-shot learning meth-ods can be quite different across datasets and seeds for the training set (Zhao et al., 2021; Schick and Schütze, 2021a).”
- Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2021, methodology): “EXAMP. setup which resembles similar studies in GPT3 prompting (Zhao et al., 2021).”
- Memory-assisted prompt editing to improve GPT-3 after deployment (2022, methodology): “Future work can employ strategies in recent literature on prompt-fine tuning (Zhao et al., 2021) to best combine fb with p e.”
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (2022, methodology): “First, using the demonstrations with gold labels significantly improves the performance over no demonstrations, 5 as it has been consistently found in much of prior work (Brown et al., 2020; Zhao et al., 2021; Liu et al., 2021).”
- Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, background): “As presented in Table 2, we found that structuring SNLI as a question-answering task achieved the best performance, similarly to Zhao et al. (2021).”
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, background): “[151] identify three pitfalls (majority label bias, recency bias and common token bias) that lead the pre-trained LMs to be biased toward certain answers when provided answered prompts.”
- Rationale-Augmented Ensembles in Language Models (2022, background): “…can be improved by sampling multiple language model outputs for ensembling, (2) prompt-order ensembling, where previous work (Lu et al., 2021; Zhao et al., 2021) has shown that task performance is sensitive to the order of the exemplars in the prompts, and (3) input-rationale ensembling,…”
- Language Models are Few-Shot Butlers (2021, background): “…the standard fine-tuning paradigm, more sophisticated approaches could be explored (Schick and Schütze, 2020) and recent works (Brown et al., 2020; Zhao et al., 2021) even suggest that scaled-up and care-fully calibrated models achieve great downstream results without requiring any parameter…”
- Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, neutral): “Similar to the observations in previous studies [5, 70, 85], the zero-shot predictions of unidirectional PLMs are quite inaccurate and directly using them as soft labels to train classifiers does not yield good results.”
- True Few-Shot Learning with Language Models (2021, background): “However, the few-shot performance of LMs is very sensitive to the textual task description [“prompt”; 3–6], order of training examples [6–8], decoding strategy [9, 10], and other hyperparameters [3, 5, 9, 11, 12], as well as the learning algorithm itself [3, 12].”
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity (2021, methodology): “An intuitive way to address this would be by calibrating the output distribution, along the lines of Zhao et al. (2021).”
- Flamingo: a Visual Language Model for Few-Shot Learning (2022, background): “However, in-context learning is known to be highly sensitive to various aspects of the demonstrations [80, 148], and its inference compute cost and absolute performance scale poorly with the number of shots beyond this low-data regime.”
- GPT Understands, Too (2021, background): “GPT-3 (Brown et al., 2020) uses in-context examples (Liu et al., 2021; Zhao et al., 2021b) as a way of prompting to transfer knowledge from pretraining to downstream tasks.”
- Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022, background): “[12] showed that the ordering of examples in the context heavily influences the model’s predictions.”