A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation

By Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, William B. Dolan. 2021

203 citations recorded by the source corpus.

Large pretrained generative models like GPT-3 often suffer from hallucinating non-existent or incorrect content, which undermines their potential merits in real applications. Existing work usually attempts to detect these hallucinations based on a corresponding oracle reference at a sentence or document level. However ground-truth references may not be readily available for many free-form text generation applications, and sentence- or document-level detection may fail to provide the fine-grained signals that would prevent fallacious content in real time. As a first step to addressing these issues, we propose a novel token-level, reference-free hallucination detection task and an associated annotated dataset named HaDeS (HAllucination DEtection dataSet). To create this dataset, we first perturb a large number of text segments extracted from English language Wikipedia, and then verify these with crowd-sourced annotations. To mitigate label imbalance during annotation, we utilize an iterative model-in-loop strategy. We conduct comprehensive data analyses and create multiple baseline models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (2023, background): “Generation models prone to generate unrelated content beyond the input data and this phenomenon has been recognized as the hallucination problem (Tian et al., 2019; Nie et al., 2019; Liu et al., 2021; Rebuffel et al., 2021) in many recent research works.”
When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories (2022, background): “Despite its impressive performance on wider tasks, large LMs often hallucinate while their outputs often look plausible, making it difficult to understand when their outputs are unreliable (Liu et al., 2022).”
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization (2021, methodology): “Inspired by recent work that studies model prediction conﬁdence (Liu et al., 2021), we examine generation probabilities for tokens of different part-of-speech (POS) tags .”
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023, background): “Further, Liu et al. (2022) introduce a hallucination detection dataset, however, texts are obtained by perturbing factual texts and thus may not reflect true LLM hallucination.”
ExpertQA: Expert-Curated Questions and Attributed Answers (2023, background): “Dedicated benchmarks to study hallucination have also been proposed (Liu et al., 2022; Li et al., 2023), but these are often synthetically generated and not based on real user queries and responses to them.”
Learning to Generate Code Sketches (2021, background): “While LMs generate realistic-looking outputs, they are known to occasionally “hallucinate” [27, 22, 23, 19], i.e. generate plausible but incorrect content.”