Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation

By Lorenz Kuhn, Yarin Gal, Sebastian Farquhar. International Conference on Learning Representations, 2023

830 citations recorded by the source corpus.

We introduce a method to measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natural language outputs of foundation models. We show that measuring uncertainty in natural language is challenging because of"semantic equivalence"-- different sentences can mean the same thing. To overcome these challenges we introduce semantic entropy -- an entropy which incorporates linguistic invariances created by shared meanings. Our method is unsupervised, uses only a single model, and requires no modifications to off-the-shelf language models. In comprehensive ablation studies we show that the semantic entropy is more predictive of model accuracy on question answering data sets than comparable baselines.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/507465f8d46489a68a527cb5304d76bdb6c31ed9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Weizhu Chen
3. Chelsea Finn
4. Sergey Levine
5. Mohit Bansal
6. Nan Duan
7. Percy Liang
8. Yelong Shen

Citations in the corpus

• Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models (2023, methodology): “As was recently argued in Kuhn et al. (2023), such approaches enable the application of uncertainty quantification measures akin to those employed in more extensively researched classification or regression contexts, but overlook the generative aspects and distinct challenges inherent to NLG.…”
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (2023, background): “In addition, beyond its simplicity, this approach also offers an extra benefit over model’s token-likelihood: the verbalized confidence is intrinsically tied to the semantic meaning of the answer instead of its syntactic or lexical form (Kuhn et al., 2023).”
• Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners (2023, methodology): “Unlike Kuhn et al. [38] who calculate the entropy among the groups to decide whether to trust the answer to a question, K NOW N O instead combines the normalized probabilities with conformal prediction to provide set-based predictions with coverage guarantees.”
• INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (2024, methodology): “Following prior work Kuhn et al. (2022); Ren et al. (2022), we evaluate the hallucination detection ability of different methods by employing them to determine whether the generation is correct or not.”
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, methodology): “We use the discrimination metric AUROC as a better measure of uncertainty for free-form generation than calibration metrics ECE or Brier score (Kuhn et al., 2023; Si et al., 2023).”
• AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “Recent work has begun to explore techniques for generating uncertainty estimates that more accurately capture correctness, including using probabilistic methods [104], finetuning the LLM to describe its own confidence [125], and sampling multiple outputs and having the LLM evaluate them [90].”
• SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023, background): “Recently, semantic uncertainty has been proposed to address uncertainty in free-form generation tasks where probabilities are attached to concepts instead of tokens (Kuhn et al., 2023).”
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback (2023, background): “…et al., 2022a; Park and Caragea, 2022; Kadavath et al., 2022; Xiao et al., 2022; Kuhn et al., 2023), finding that combining large pre-trained LMs with temperature scaling (Guo et al., 2017) produces very well-2 calibrated predictions (Kadavath et al., 2022; Xiao et al., 2022; Kuhn et al., 2023).”
• Empowering Biomedical Discovery with AI Agents (2024, neutral): “Challenges remain and may, in some cases, be amplified when multi-agent systems become available (Figure 6).”
• Better Instruction-Following Through Minimum Bayes Risk (2024, unknown): “Kuhn et al. (2023) propose an MBR-esque algorithm that uses dense embedders and clustering to measure semantic uncertainty.”