COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics

By Lianhui Qin, Sean Welleck, Daniel Khashabi, Yejin Choi. 2022

206 citations recorded by the source corpus.

Many applications of text generation require incorporating different constraints to control the semantics or style of generated text. These constraints can be hard (e.g., ensuring certain keywords are included in the output) and soft (e.g., contextualizing the output with the left- or right-hand context). In this paper, we present Energy-based Constrained Decoding with Langevin Dynamics (COLD), a decoding framework which unifies constrained generation as specifying constraints through an energy function, then performing efficient differentiable reasoning over the constraints through gradient-based sampling. COLD decoding is a flexible framework that can be applied directly to off-the-shelf left-to-right language models without the need for any task-specific fine-tuning, as demonstrated through three challenging text generation applications: lexically-constrained generation, abductive reasoning, and counterfactual reasoning. Our experiments on these constrained generation tasks point to the effectiveness of our approach, both in terms of automatic and human evaluation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Aligning Language Models with Preferences through f-divergence Minimization (2023, methodology): “The first family of approaches includes guided decoding methods (Dathathri et al., 2020; Qin et al., 2022), Monte Carlo sampling techniques such as rejection sampling to sample from simple distributions like pGDC bin (Roller et al., 2021; Ziegler et al., 2022), and Quasi Rejection Sampling (QRS)…”
NaturalProver: Grounded Mathematical Proof Generation with Language Models (2022, background): “Overall, naively ﬁne-tuning GPT-3 on theorem-proof examples alone is suboptimal for proof generation.”
A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, methodology): “Similarly, COLD Decoding [106], an energy-based and iterative CTG approach, formulates the controlled text generation task as sampling from an energy-based model using Langevin Dynamics, without the need for any task-specific fine-tuning.”
Universal and Transferable Adversarial Attacks on Aligned Language Models (2023, unknown): “Alternatively, recent work also leverages Langevin dynamics sampling to sample from discrete prompts while leveraging continuous embeddings [Qin et al., 2022].”
Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts (2021, background): “The recent works have used additional signals such as domain-specific constraints (Qin et al., 2020; Khot et al., 2021; Qin et al., 2022) to alleviate these challenges.”
Gradient-based Constrained Sampling from Language Models (2022, methodology): “…predicting the sequence length (Wang et al., 2021), generating multiple outputs with varying lengths and reranking (Guo et al., 2019), continuing generating autoregressively to finish a sentence after a fixed length output is generated (Qin et al., 2022) of all which have shown promising results.”
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022, methodology): “Fluent vs. Gibberish Prompts We propose to optimize fluent prompts with top-k filtering (Qin et al., 2022).”
RankGen: Improving Text Generation with Large Ranking Models (2022, background): “…et al., 2019; Deng et al., 2020), machine translation (Shen et al., 2004; Lee et al., 2021; Bhattacharyya et al., 2021), constrained generation (Qin et al., 2022; Mireshghallah et al., 2022), and models for speciﬁc attributes like style (Dathathri et al., 2020; Yang and Klein, 2021), length…”
Generating Sequences by Learning to Self-Correct (2022, methodology): “, 2022), or refining continuous variables (Lee et al., 2020; Li et al., 2022; Qin et al., 2022).”