ColdGANs: Taming Language GANs with Cautious Sampling Strategies

By Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano. 2020

18 citations recorded by the source corpus.

Training regimes based on Maximum Likelihood Estimation (MLE) suffer from known limitations, often leading to poorly generated text sequences. At the root of these limitations is the mismatch between training and inference, i.e. the so-called exposure bias, exacerbated by considering only the reference texts as correct, while in practice several alternative formulations could be as good. Generative Adversarial Networks (GANs) can mitigate those limitations but the discrete nature of text has hindered their application to language generation: the approaches proposed so far, based on Reinforcement Learning, have been shown to underperform MLE. Departing from previous works, we analyze the exploration step in GANs applied to text generation, and show how classical sampling results in unstable training. We propose to consider alternative exploration strategies in a GAN framework that we name ColdGANs, where we force the sampling to be close to the distribution modes to get smoother learning dynamics. For the first time, to the best of our knowledge, the proposed language GANs compare favorably to MLE, and obtain improvements over the state-of-the-art on three generative tasks, namely unconditional text generation, question generation, and abstractive summarization.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Generative Models

Citations in the corpus

Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, methodology): “Since reward model accuracy can quickly degrade if not exposed to this new sample distribution, i.e., from hyper-specialization (Scialom et al., 2020b), it is important before a new Llama 2-Chat tuning iteration to gather new preference data using the latest Llama 2-Chat iterations.”
Pre-Trained Language Models for Text Generation: A Survey (2022, unknown): “ColdGANs [210] explored the unconditional language generation.”
Residual Energy-Based Models for Text (2021, background): “…results do not necessarily imply better generations (Zhang et al., 2020; Hashimoto et al., 2019): for example, approaches exhibiting mode-dropping problems such as language GANs might score terribly in terms of perplexity but do well in terms of generation (Yu et al., 2017; Scialom et al., 2020).”