Contrastive Decoding: Open-ended Text Generation as Optimization

By Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B. Hashimoto, Luke S. Zettlemoyer, Michael Lewis. Annual Meeting of the Association for Computational Linguistics, 2022

688 citations recorded by the source corpus.

Given a language model (LM), maximum probability is a poor decoding objective for open-ended generation, because it produces short and repetitive text. On the other hand, sampling can often produce incoherent text that drifts from the original topics. We propose contrastive decoding (CD), a reliable decoding approach that optimizes a contrastive objective subject to a plausibility constraint. The contrastive objective returns the difference between the likelihood under a large LM (called the expert, e.g. OPT-13B) and a small LM (called the amateur, e.g. OPT-125M), and the constraint ensures that the outputs are plausible. CD is inspired by the fact that the failures of larger LMs (e.g., repetition, inco- herence) are even more prevalent in smaller LMs, and that this difference signals which texts should be preferred. CD requires zero additional training, and produces higher quality text than decoding from the larger LM alone. It also works across model scales (OPT-13B and GPT2-1.5B) and significantly outperforms four strong decoding algorithms (e.g., nucleus, top-k) in automatic and human evaluations across wikipedia, news and story domains.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/88b62496cbc52072bfa8f4b29d172b0477b701bc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding (2024, methodology): “For instance, Yuan et al. (2023) combined Speculative Decoding with Contrastive Decoding (Li et al., 2023), which not only speeds up the inference but also substantially improves the generation quality.”
RankGen: Improving Text Generation with Large Ranking Models (2022, background): “", for example "PG19-XL-I N B OOK ". and Collier, 2022), contrastive decoding (Li et al., 2022), and eta sampling (Hewitt et al., 2022).”
Grounding Language Models to Images for Multimodal Generation (2023, background): “When tasked to generate text, these large models also often exhibit failure modes such as neural text degeneration and generation of incoherent and repetitive text (Holtzman et al., 2020; Li et al., 2022).”
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2023, methodology): “This provides a new form of contrastive decoding (Li et al., 2022), which effectively downweights the prior knowledge when more relevant contextual information is provided.”
Knowledge Conflicts for LLMs: A Survey (2024, positive): “Knowledge plug-in approaches, such as Continuously-updated QA (Lee et al., 2022a), use plug-and-play modules to store updated knowledge, solving knowledge conflicts without affecting the original model.”
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (2024, neutral): “(see Appendix A.1.2 for details).”
Pre-Trained Language Models for Text Generation: A Survey (2022, background): “Contrastive decoding [135] prefers the token that large LMs assign more probability than small LMs since larger LMs put more mass on desirable token.”
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (2023, background): “, 2023) (7B, 13B, 33B, 65B) and compare them with three baselines: 1) original decoding (greedy decoding or sampling depending on the tasks), 2) Contrastive Decoding (CD) (Li et al., 2022), where LLaMA-7B serves as the amateur model, while LLaMA13B/33B/65B act as expert models, and 3) Inference Time”
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (2023, methodology): “To address this issue, we follow Li et al. [37] to implement an adaptive plausibility constraint that is contingent upon the confidence level associated with the output distribution with original visual inputs: where V is the output vocabulary of LVLMs and β is a hyperparameter in [0 , 1] for…”
Hallucination of Multimodal Large Language Models: A Survey (2024, neutral): “The lack of standardized benchmarks and evaluation metrics poses significant challenges in assessing the degree of hallucination in MLLMs.”
Text2Motion: from natural language instructions to feasible plans (2023, methodology): “While we mitigate such failures by combining greedy-search and shooting in the hybrid Text2Motion algorithm, leveraging calibration techniques to increase the reliability LLM likelihoods [64, 65] may improve the performance search-based planning over long-horizons.”
Training Trajectories of Language Models Across Scales (2022, methodology): “Our decoding approach is similar to the contrastive decoding method (CD) proposed in Li et al. (2022), though initially for completely different purposes.”