RankGen: Improving Text Generation with Large Ranking Models

By Kalpesh Krishna, Yapei Chang, John Wieting, Mohit Iyyer. 2022

87 citations recorded by the source corpus.

Given an input sequence (or prefix), modern language models often assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as such, model-generated text also contains such artifacts. To address these issues we present RankGen, a 1.2B parameter encoder model for English that scores model generations given a prefix. RankGen can be flexibly incorporated as a scoring function in beam search and used to decode from any pretrained language model. We train RankGen using large-scale contrastive learning to map a prefix close to the ground-truth sequence that follows it and far away from two types of negatives: (1) random sequences from the same document as the prefix, and (2) sequences generated from a large language model conditioned on the prefix. Experiments across four different language models (345M-11B parameters) and two domains show that RankGen significantly outperforms decoding algorithms like nucleus, top-k, and typical sampling on both automatic metrics (85.0 vs 77.3 MAUVE) as well as human evaluations with English writers (74.5% human preference over nucleus sampling). Analysis reveals that RankGen outputs are more relevant to the prefix and improve continuity and coherence compared to baselines. We release our model checkpoints, code, and human preference data with explanations to facilitate future research.

Subfields: Artificial Intelligence, Large Language Models, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Natural Language Generation

Citations in the corpus

An Empirical Study of the Non-Determinism of ChatGPT in Code Generation (2023, unknown): “The randomness in LLMs typically comes from the sampling methods used during text generation, such as top-k sampling or nucleus sampling [31, 49].”
Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning (2022, background): “In particular Krishna et al. (2022) shows the robustness of using a contrastive learning based signal to aid in natural language generation.”
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP (2022, unknown): “…we can invoke the RM via rank to ﬁnd the prediction that is most grounded in a retrieved contexts (i.e., most similar to the concatenation of the retrieved passages) or, given an RM that can score completions (Krishna et al., 2022), simply the prediction that has the highest score given the prompt.”
Training Trajectories of Language Models Across Scales (2022, methodology): “Our generation protocol is slightly different from the standard open-ended generation practices in that we only use 5 tokens as prompts for generation, while usually at least 128 tokens are used (Krishna et al., 2022; Su and Collier, 2022; Li et al., 2022).”
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense (2023, methodology): “Moreover, we evaluate larger language models like GPT3.”