How Decoding Strategies Affect the Verifiability of Generated Text

By Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktaschel, Vassilis Plachouras, Fabrizio Silvestri, Sebastian Riedel. 2019

60 citations recorded by the source corpus.

Recent progress in pre-trained language models led to systems that are able to generate text of an increasingly high quality. While several works have investigated the fluency and grammatical correctness of such models, it is still unclear to which extent the generated text is consistent with factual world knowledge. Here, we go beyond fluency and also investigate the verifiability of text generated by state-of-the-art pre-trained language models. A generated sentence is verifiable if it can be corroborated or disproved by Wikipedia, and we find that the verifiability of generated text strongly depends on the decoding strategy. In particular, we discover a tradeoff between factuality (i.e., the ability of generating Wikipedia corroborated text) and repetitiveness. While decoding strategies such as top-k and nucleus sampling lead to less repetitive generations, they also produce less verifiable text. Based on these finding, we introduce a simple and effective decoding strategy which, in comparison to previously used decoding strategies, produces less repetitive and more verifiable text.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, background): “Similar to Li et al. [29], Vijayakumar et al. [53] and Massarelli et al. [35], we also investigate the diversity of generations by calculating the ratio of distinct ngrams to total ngrams generated by different models. diverse than RAG-Token generations, and both generate signiﬁcantly more diverse…”
Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “However, such approaches have been shown to be unreliable because of their tendency to hallu-cinate non-factual content (Massarelli et al., 2019; Metzler et al., 2021; Ji et al., 2022).”
Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, background): “Inspired by work in other generation tasks (Holtzman et al., 2020; Massarelli et al., 2020; Holtzman et al., 2021), we hypothesize that equally or more informative explanations can be generated by sampling stochastically.”
Recipes for Building an Open-Domain Chatbot (2020, background): “Firstly, even our best models still make mistakes: although relatively rarely, they i) contradict or repeat themselves on occasion, ii) tend to repeat the same phrases in separate conversations, and iii) hallucinate knowledge as seen in other generative systems (Massarelli et al., 2019).”
Evaluation of Text Generation: A Survey (2020, background): “Massarelli et al. (2019) introduce a new approach for generating text that is factually consistent with the knowledge source.”
GO FIGURE: A Meta Evaluation of Factuality in Summarization (2020, background): “Recent efforts by NLP researchers have drawn attention to the issue of factual errors and hallucinations in the output of neural summarization models (Cao et al., 2018; Massarelli et al., 2019; Zhao et al., 2020; Falke et al., 2019b; Goodrich et al., 2019; Celikyilmaz et al., 2020).”
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022, background): “Better decoding strategies are needed to ensure the label’s correctness while preserving the dataset diversity (Massarelli et al., 2020).”
Knowledge Conflicts for LLMs: A Survey (2024, unknown): “Due to the potential limitations inherent in the editing method, the modified knowledge cannot be generalized effectively.”
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation? (2019, background): “As shown by recent studies, there exists important problems other than exposure bias for the current NLG models, such as the likelihood trap (Holtzman et al., 2020), factuality (Massarelli et al., 2019; He et al., 2021), or robustness (Cheng et al., 2018; Kassner and Schütze, 2019), etc.”
RankGen: Improving Text Generation with Large Ranking Models (2022, background): “…as top-k (Fan et al., 2018), nucleus (Holtzman et al., 2020), and typical sampling (Meister et al., 2022) alleviate these issues, they can also produce text with inconsistencies, hallucinations, factual errors, or commonsense issues (Massarelli et al., 2020; Dou et al., 2022; Krishna et al., 2021).”
Video Understanding as Machine Translation (2020, background): “, for captioning) we decode open-ended text [29].”
Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation (2021, methodology): “Model size: We use the four variants of GPT2 (Radford et al., 2019) based language models: small, medium, large, XL. Decoding strategies: For decoding we use Nucleus sampling (NS) (Holtzman et al., 2020) with p=0.9; Beam-Search (BS) with a beam size of 5, and Delayed Beam-Search (DBS) (Massarelli et”
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers (2021, background): “Property-specific statistics such as the amount of repetition [26, 59], verifiability [40], or termination [58] are orthogonal to MAUVE, which provides a summary of the overall gap between P and Q rather than focusing on an individual property.”