FUDGE: Controlled Text Generation With Future Discriminators

By Kevin Yang, Daniel Klein. 2021

448 citations recorded by the source corpus.

We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation. Given a pre-existing model G for generating text from a distribution of interest, FUDGE enables conditioning on a desired attribute a (for example, formality) while requiring access only to G’s output logits. FUDGE learns an attribute predictor operating on a partial sequence, and uses this predictor’s outputs to adjust G’s original probabilities. We show that FUDGE models terms corresponding to a Bayesian decomposition of the conditional distribution of G given attribute a. Moreover, FUDGE can easily compose predictors for multiple desired attributes. We evaluate FUDGE on three tasks — couplet completion in poetry, topic control in language generation, and formality change in machine translation — and observe gains in all three tasks.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, methodology): “Motivated by PPLM, FUDGE (Yang and Klein, 2021) introduces a future discriminator to predict whether the ongoing generation text would conform to the desired attribute, greatly accelerating decoding.”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “Studies on these tasks have explored (1) modifying attention mechanisms to specifically capture relationships between entities (Nan et al., 2021; Zhu et al., 2021), (2) improving coreference to mitigate gender bias in translation (Levy et al., 2021), and (3) using text entailment to develop loss…”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, neutral): “This is similar in spirit to previous work on using cooperative discriminators on uncontrolled LLMs (Holtzman et al., 2018; Krause et al., 2020; Yang and Klein, 2021; Liu et al., 2021a), yet in this work our LLM is controlled in an adversarial way by prompting and by a classifier.”
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, background): “Adapting PLMs to generate texts of specific attributes can also be realized at inference time without any further training of the PLMs [10, 26, 27, 32, 47, 75].”
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021, methodology): “Contemporaneous with our work, FUDGE (Yang and Klein, 2021) trains classifiers on partial sequences to predict whether an attribute will be satisfied in the future, and uses Bayesian factorization to obtain the attribute-conditioned probability distribution.”
Diffusion-LM Improves Controllable Text Generation (2022, methodology): “To evaluate the fluency of the generated text, we follow the prior works [55, 6] and feed the generated text to a teacher LM (i.”
Director: Generator-Classifiers For Supervised Language Modeling (2022, positive): “(3) In methods such as FUDGE, a separate classiﬁer is trained, but it is not efﬁcient because the classiﬁer needs to be evaluated for each candidate token x t ∈ V in the vocabulary at every time step t .”
Gradient-based Constrained Sampling from Language Models (2022, methodology): “…in an autoregressive (i.e., left-to-right) decoding strategy like beam search or sampling is challenging, since the constraints are defined conceptually on the whole output sequence and are hard to evaluate accurately only on the generated prefix (Yang and Klein, 2021b; Liu et al., 2021a).”
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning (2022, background): “…phrases (Chan et al., 2021)) or both (Khal-ifa et al., 2021); at inference time, control over desired attributes can also be enforced without updating the PLM parameters (Dathathri et al., 2020; Krause et al., 2021; Kumar et al., 2021; Liu et al., 2021a; Pascual et al., 2021; Yang & Klein, 2021).”
RankGen: Improving Text Generation with Large Ranking Models (2022, background): “…al., 2004; Lee et al., 2021; Bhattacharyya et al., 2021), constrained generation (Qin et al., 2022; Mireshghallah et al., 2022), and models for speciﬁc attributes like style (Dathathri et al., 2020; Yang and Klein, 2021), length (Li et al., 2017), or repetition & relevance (Holtzman et al., 2018).”
CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation (2022, unknown): “Recent works re-sort to decoding-time methods and directly make pre-trained models generate texts towards desired attributes during inference, including PPLM (Dathathri et al., 2020), GeDi (Krause et al., 2020), FUDGE (Yang and Klein, 2021) and DEXPERTS (Liu et al., 2021a).”
Systematic Rectification of Language Models via Dead-end Analysis (2023, background): “However, PPLM is known to be computationally expensive and slow Gehman et al. (2020); Yang & Klein (2021).”
RRHF: Rank Responses to Align Language Models with Human Feedback without tears (2023, background): “The research field of aligning with human preference is related to controlled language generation which field has rich literature [13, 31, 8].”
Offline RL for Natural Language Generation with Implicit Language Q Learning (2022, methodology): “We compare ILQL to: (1) ILQL (utterance): a per-utterance adaptation of ILQL that is nearly identical to ILQL, except that it removes the conservatism loss term and performs Bellman backups at the utterance level instead of the token level; (2) SARSA (utterance): a per-utterance version of SARSA, wh”
Controlled Text Generation as Continuous Optimization with Multiple Constraints (2021, methodology): “We use the recommended hyperparameters by Yang and Klein [72] for decoding.”
A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, methodology): “…167] Post-process efficient training and inefficient inference; lower text quality; better controllability Guided Strategy [24, 62, 74, 117, 150, 151] [73, 89, 96, 106] Trainable Strategy [25, 59] We list the main characteristics for each categor from a macro perspective, and representative…”
Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review (2023, methodology): “Throughout this generation process, an OPT-based Future Discriminators for Generation (FUDGE) [121] detailed controller plays a crucial role in maintaining relevance.”
Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, background): “, sentiment) [127, 263], there is little progress on complex, fine-grained controls (e.”