Discourse-Aware Neural Rewards for Coherent Text Generation

By Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, Yejin Choi. 2018

82 citations recorded by the source corpus.

In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text. In particular, we propose to learn neural rewards to model cross-sentence ordering as a means to approximate desired discourse structure. Empirical results demonstrate that a generator trained with the learned reward produces more coherent and less repetitive text than models trained with cross-entropy or with reinforcement learning with commonly used scores as rewards.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Towards Coherent and Cohesive Long-form Text Generation (2018, background): “The most relevant to our work is Bosselut et al. (2018), which promotes text generation in the correct order, and discourages in its reverse order us-ing rewards.”
Evaluation of Text Generation: A Survey (2020, background): “, 2018), recipe generation (Bosselut et al., 2018), image captioning (Lan et al.”
Cooperative Generator-Discriminator Networks for Abstractive Summarization with Narrative Flow (2019, background): “A number of solutions have been proposed in recent years for improving coherence in text generation, including global-tracking of entities and discourseaware neural rewards (Kiddon et al., 2016; Bosselut et al., 2018; Holtzman et al., 2018; Fan et al., 2018b).”
Faking Fake News for Real Fake News Detection: Propaganda-Loaded Training Data Generation (2022, methodology): “The second sub-goal is accomplished using self-critical sequence training (Rennie et al., 2017; Bosselut et al., 2018) with a Natural Language Inference (NLI) component, which is used as a reward function for generation that meets this sub-goal.”
Robust Text-to-SQL Generation with Execution-Guided Decoding (2018, methodology): “Sequence-Level Objectives Several recently introduced techniques use reinforcement learning to incorporate sequence-level objectives into training, such as BLEU in coherent text generation (Bosselut et al. 2018), semantic coherence in dialogue generation (Li et al.”
The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation (2021, background): “Each paper was published between 2018 and 2020 at ACL, NAACL, or EMNLP, and we exclude papers that use AMT to evaluate more wellestablished generation tasks like machine translation, or summarization.3 Unlike previous surveys of evaluating generated text (Çelikyilmaz et al., 2020; van der Lee et al.”
Learning to Generalize from Sparse and Underspecified Rewards (2019, background): “…the automatic discovery of goal speciﬁ-cations (Xie et al., 2018; Bahdanau et al., 2019), text generation tasks (Wang et al., 2018; Wu et al., 2017; Bosselut et al., 2018) and the optimization of reward functions ( e.g., Gleave & Habryka (2018); Fu et al. (2019); Shi et al. (2018)) via inverse RL.”
DAGN: Discourse-Aware Graph Network for Logical Reasoning (2021, background): “, 2018), and coherent text generation (Wang et al., 2020; Bosselut et al., 2018).”
AutoSurvey2: Empowering Researchers with Next Level Automated Literature Surveys (2025, unknown): “Building upon recent advances in automated literature review generation (Bosselut et al., 2018; Cho et al., 2018; Wang et al., 2019), the system integrates large language models (LLMs), semantic vector retrieval, and a modular graph-based execution mechanism to emulate the scholarly writing process…”
AutoSurvey: Large Language Models Can Automatically Write Surveys (2024, unknown): “Compared to traditional methods, which primarily focus on training models to generate linguistically coherent text [39, 40], the emergency of large language models (LLMs) has opened up new possibilities for automated writing, drawing more attention to broader aspects like faithfulness, logical…”
Bloom-Eval: A Hierarchical Evaluation Benchmark for Automatic Survey Generation Based on Bloom's Taxonomy (2026, unknown): “…Chen and Zhuge, 2019; Wang et al., 2019; Lu et al., 2020; Jin et al., 2020; Xing et al., 2020) and long-form text generation (Fan et al., 2018; Bosselut et al., 2018; Cho et al., 2019; Mao et al., 2022; Kryscinski et al., 2022; Chang et al., 2024), where researchers have explored diverse…”