Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models

By Zhen Lin, Shubhendu Trivedi, Jimeng Sun. Trans. Mach. Learn. Res., 2023

322 citations recorded by the source corpus.

Large language models (LLMs) specializing in natural language generation (NLG) have recently started exhibiting promising capabilities across a variety of domains. However, gauging the trustworthiness of responses generated by LLMs remains an open challenge, with limited research on uncertainty quantification (UQ) for NLG. Furthermore, existing literature typically assumes white-box access to language models, which is becoming unrealistic either due to the closed-source nature of the latest LLMs or computational constraints. In this work, we investigate UQ in NLG for *black-box* LLMs. We first differentiate *uncertainty* vs *confidence*: the former refers to the ``dispersion'' of the potential predictions for a fixed input, and the latter refers to the confidence on a particular prediction/generation. We then propose and compare several confidence/uncertainty measures, applying them to *selective NLG* where unreliable results could either be ignored or yielded for further assessment. Experiments were carried out with several popular LLMs on question-answering datasets (for evaluation purposes). Results reveal that a simple measure for the semantic dispersion can be a reliable predictor of the quality of LLM responses, providing valuable insights for practitioners on uncertainty management when adopting LLMs. The code to replicate our experiments is available at https://github.com/zlin7/UQ-NLG.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/ad934a9344f68fcc0b9aa704102aa48c39c5b591

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiliang Tang
2. Dawei Yin
3. Hui Liu
4. Wenqi Fan
5. Wei Jin
6. Mingyuan Tao
7. Chao Chen
8. Yue Wu

Citations in the corpus

• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, unknown): “Uncertainty can assist users in determining when to trust LLMs. Provided that the uncertainty of LLM responses can be accurately characterized, users can filter out or rectify LLMs’ claims with high uncertainty since such claims are more prone to be fabricated ones (Lin et al., 2023).”
• INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (2024, methodology): “AU-ROC is a popular metric to evaluate the quality of a binary classifier and uncertainty measure (Ren et al., 2022; Lin et al., 2023).”
• Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs (2023, methodology): “Following our proposed two pipelines, i.e., LLMs as the En-hancers and LLMs as the Predictors, we review existing works in this section.”
• Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation (2025, unknown): “On the other hand, Monte Carlo sampling-based approaches (Zhang, 2020; Malinin & Gales, 2021; Lin et al., 2024; Lakshminarayanan et al., 2017) have shown promise in estimating uncertainty in LLMs.”