Factual Probing Is [MASK]: Learning vs. Learning to Recall

By Zexuan Zhong, Dan Friedman, Danqi Chen. 2021

460 citations recorded by the source corpus.

Petroni et al. (2019) demonstrated that it is possible to retrieve world facts from a pre-trained language model by expressing them as cloze-style prompts and interpret the model’s prediction accuracy as a lower bound on the amount of factual information it encodes. Subsequent work has attempted to tighten the estimate by searching for better prompts, using a disjoint set of facts as training data. In this work, we make two complementary contributions to better understand these factual probing techniques. First, we propose OptiPrompt, a novel and efficient method which directly optimizes in continuous embedding space. We find this simple method is able to predict an additional 6.4% of facts in the LAMA benchmark. Second, we raise a more important question: Can we really interpret these probing results as a lower bound? Is it possible that these prompt-search methods learn from the training data too? We find, somewhat surprisingly, that the training data used by these methods contains certain regularities of the underlying fact distribution, and all the existing prompt methods, including ours, are able to exploit them for better fact prediction. We conduct a set of control experiments to disentangle “learning” from “learning to recall”, providing a more detailed picture of what different prompts can reveal about pre-trained language models.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2021, background): “We consider: • A UTO P ROMPT : Following (Shin et al., 2020), we search for discrete tokens to use in the input instead of manually-designed patterns.”
Making Pre-trained Language Models Better Few-shot Learners (2021, background): “, 2020), or require a large number of examples for gradientguided search (Shin et al., 2020; Zhong et al., 2021).”
True Few-Shot Learning with Language Models (2021, methodology): “In what follows, we test on LAMA [17], a benchmark for retrieving facts with LMs, for which prior work has developed many strategies for designing prompts [4, 54–56].”
Locating and Editing Factual Associations in GPT (2022, background): “Later work showed that knowledge extraction can be improved by diversifying the prompts (Jiang et al., 2020; Zhong et al., 2021), or by fine-tuning a model on open-domain textual facts (Roberts et al.”
GPT Understands, Too (2021, background): “Results show that P-Tuning can be effectively combined with existing discrete patterns and achieve further performance improvement. cusing on improving knowledge probing (Qin and Eisner, 2021; Zhong et al., 2021).”
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks (2021, background): “, 2020), continuous prompts (Liu et al., 2021b; Li and Liang, 2021; Lester et al., 2021; Qin and Eisner, 2021; Zhong et al., 2021), tuning bias (Logan IV et al.”
Learning To Retrieve Prompts for In-Context Learning (2021, methodology): “…et al., 2019; Schick and Schütze, 2021), decoding meth-ods (Min et al., 2021; Zhao et al., 2021; Holtzman et al., 2021), and methods for extracting either hard (Shin et al., 2020; Haviv et al., 2021) or soft (Li and Liang, 2021; Zhong et al., 2021; Qin and Eisner, 2021) prompts automatically.”
Pre-trained models for natural language processing: A survey (2020, background): “[204] found that the optimized continuous prompt can outperform concrete prompts (including manual [129], mined (LPAQA [201]), and gradient-searched (AutoPrompt [71]) prompts) on relational tasks.”
PPT: Pre-trained Prompt Tuning for Few-shot Learning (2021, background): “To overcome the shortcomings of discrete spaces, Li and Liang (2021); Liu et al. (2021); Han et al. (2021b); Hambardzumyan et al. (2021); Zhong et al. (2021b) explore to combine hard prompts and soft prompts.”
Noisy Channel Language Model Prompting for Few-Shot Text Classification (2021, background): “…calibration (Zhao et al., 2021; Holtz-man et al., 2021), or learning better prompts, either in a discrete space (Shin et al., 2020; Jiang et al., 2020; Gao et al., 2021) or in a continuous space (Li and Liang, 2021; Lester et al., 2021; Liu et al., 2021; Zhong et al., 2021; Qin and Eisner, 2021).”
Black-Box Tuning for Language-Model-as-a-Service (2022, background): “Note that the prompt is not necessarily discrete, it can also be optimized efﬁciently in continuous space with gradient descent (Li & Liang, 2021; Hambardzumyan et al., 2021; Qin & Eisner, 2021; Liu et al., 2021b; Zhong et al., 2021).”
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers (2024, unknown): “Selection : select a certain number of prompts from current population as parent prompts p r 1 , . . . , p r k ∼ P t − 1”
Natural Language to Code Translation with Execution (2022, background): “…models also allow us 2 While existing work on prompting language models usually requires a task-speciﬁc design of prompts (Shin et al., 2020; Zhong et al., 2021; Gao et al., 2021, inter alia ), we ﬁnd to have the associated probability of generating each generated token w i conditioned on…”
Prompt Distribution Learning (2022, background): “Recently, some works [12,22,24,48], also known as prompt tuning , attempt to learn soft (continuous) prompts directly instead of searching for discrete prompts.”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, unknown): “While significant efforts have been undertaken in exploring their knowledge storage (Geva et al., 2021; Meng et al., 2022)and probing (Petroni et al., 2019; Zhong et al., 2021; Yu et al., 2023c), the exact mechanism by which LLMs capture the factual knowledge remains elusive.”
Exploring Visual Prompts for Adapting Large-Scale Models (2022, methodology): “Currently, prompting methods are dominantly NLP-speciﬁc [4, 40–42, 13, 30, 27, 36, 14, 48, 15, 26], despite the fact that the framework serves a general purpose: adapt a frozen pre-trained model by modifying the data space .”