Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?

By Lena Reed, Shereen Oraby, Marilyn A. Walker. 2018

37 citations recorded by the source corpus.

Responses in task-oriented dialogue systems often realize multiple propositions whose ultimate form depends on the use of sentence planning and discourse structuring operations. For example a recommendation may consist of an explicitly evaluative utterance e.g. Chanpen Thai is the best option, along with content related by the justification discourse relation, e.g. It has great food and service, that combines multiple propositions into a single phrase. While neural generation methods integrate sentence planning and surface realization in one end-to-end learning framework, previous work has not shown that neural generators can: (1) perform common sentence planning and discourse structuring operations; (2) make decisions as to whether to realize content in a single sentence or over multiple sentences; (3) generalize sentence planning and discourse relation operations beyond what was seen in training. We systematically create large training corpora that exhibit particular sentence planning operations and then test neural models to see what they learn. We compare models without explicit latent variables for sentence planning with ones that provide explicit supervision during training. We show that only the models with additional supervision can reproduce sentence planning and discourse operations and generalize to situations unseen in training.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Semantic Noise Matters for Neural Natural Language Generation (2019, methodology): “To produce a cleaned version of the E2E data, we used the original human textual references, but paired them with correctly matching MRs. 4 To this end, we reimplemented the slot matching script of Reed et al. (2018), which tags MR slots and values using regular expressions.”
Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge (2019, methodology): “26 We based the patterns for the individual attribute-value pairs on Reed, Oraby, and Walker (2018)’s script and manually enhanced them using the first 500 instances of the E2E development set.”
Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue (2019, methodology): “In a closely related work, Reed et al. (2018), the authors add control tokens (to indicate contrast and sentence structure) to a flat input MR, and show that these can be effectively used to control structure.”
Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference (2020, methodology): “The closest commonly used metric is the slot error rate , which is typically based on pattern matching tailored for a given dataset (Reed et al., 2018; Mi et al., 2019; Dušek et al., 2020).”
AggGen: Ordering and Aggregating while Generating (2021, background): “Factual Correctness Evaluation tests if the generated text corresponds to the input triples (Wen et al., 2015b; Reed et al., 2018; Dušek et al., 2020).”
Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation (2020, methodology): “More recent methods use Seq2Seq architecture (Sutskever et al., 2014) to combine planning and realization into an end-to-end network and have achieved the state-of-the-art on a variety of generation tasks (Lebret et al., 2016; Trisedya et al., 2018; Juraska et al., 2018; Reed et al., 2018).”
Variational Template Machine for Data-to-Text Generation (2020, methodology): “For the model trained on SPNLG dataset, the dimension of latent template variable is set as 64, and the dimension of latent content variable is set as 100.”
TaTa: A Multilingual Table-to-Text Dataset for African Languages (2022, background): “The downside of involving crowdworkers in the language generation steps is that outputs can be unnaturally phrased compared to naturally occurring scraped descriptions; studies on translationese in machine translation (Tirkkonen-Condit, 2002; Bizzoni et al., 2020) highlight potential negative…”
Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020, background): “The finer-grained segmentation might be useful if the focus is on modeling the detailed discourse structure instead of the information accuracy (Reed et al., 2018; Balakrishnan et al., 2019), which we leave for future work.”
A Tree-to-Sequence Model for Neural NLG in Task-Oriented Dialog (2019, methodology): “Previous work on AMR and WebNLG (Beck et al., 2018; Song et al., 2018; Marcheggiani and Perez-Beltrachini, 2018) has demonstrated improvements over Seq2Seq models by using graph-to-sequence models; while similar in principle, these works do not explore the use of hierarchical content plans as…”
Improving Text-to-Text Pre-trained Models for the Graph-to-Text Task (2020, background): “There are also works on adding semantic supervi-sions (Reed et al., 2018; Balakrishnan et al., 2019), however, it requires extensive efforts to annotate discourse relations in each reference text.”
Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization (2021, background): “, topics [25], sentence types [36], personas [19], emotions [43, 58], documents [30], or knowledge graphs [17, 27, 54, 59], augmenting the model itself [53, 57], or modifying the loss function [18].”