Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation

By Jingjing Xu, Xuancheng Ren, Junyang Lin, Xu Sun. 2018

152 citations recorded by the source corpus.

Existing text generation methods tend to produce repeated and ”boring” expressions. To tackle this problem, we propose a new text generation model, called Diversity-Promoting Generative Adversarial Network (DP-GAN). The proposed model assigns low reward for repeatedly generated text and high reward for ”novel” and fluent text, encouraging the generator to produce diverse and informative text. Moreover, we propose a novel language-model based discriminator, which can better distinguish novel text from repeated text without the saturation problem compared with existing classifier-based discriminators. The experimental results on review generation and dialogue generation tasks demonstrate that our model can generate substantially more diverse and informative text than existing baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

The Curious Case of Neural Text Degeneration (2019, background): “One of the most prominent recent research directions in open-ended text generation has been using generative adversarial networks (GANs; Yu et al., 2017a; Xu et al., 2018).”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “have become the most popular choice for dialog generation. Other frameworks, including conditional variational autoencoder (CVAE) [26, 56, 121, 124, 184, 185] and generative adversarial network (GAN) [64, 161], are also applied to dialog generation. Very recently, Transformer-based language models p”
A Discriminator Improves Unconditional Text Generation without Updating the Generator (2020, unknown): “of language GANs such as, DpGAN (Xu et al., 2018), maskGAN(Fedus et al.”
Comparison of Diverse Decoding Methods from Conditional Language Models (2019, background): “Xu et al. (2018) also uses an adversarial loss; their loss function rewards fluent text and penalizes repetitive text.”
Focus Attention: Promoting Faithfulness and Diversity in Summarization (2021, background): “For conditional text generation, most work focuses on generating diverse questions (Narayan et al., 2016; Dong et al., 2017; Sultan et al., 2020; Wang et al., 2020b) or paraphrases (Li et al., 2016b; Dai et al., 2017; Xu et al., 2018; Cao and Wan, 2020).”
Trading Off Diversity and Quality in Natural Language Generation (2020, background): “Zhang et al. (2018) and Xu et al. (2018) use adversarial learning methods to encourage diversity.”
Generating Counter Narratives against Online Hate Speech: Data and Strategies (2020, methodology): “count of distinct ngrams (Xu et al., 2018; Li et al., 2016b) or the standard type/token ratio (Richards, 1987) since it allows us to compare corpora of diverse sizes by averaging the statistics collected on a sliding window of 1000 words.”