A Deep Ensemble Model with Slot Alignment for Sequence-to-Sequence Natural Language Generation

By Juraj Juraska, Panagiotis Karagiannis, Kevin K. Bowden, Marilyn A. Walker. 2018

91 citations recorded by the source corpus.

Natural language generation lies at the core of generative dialogue systems and conversational agents. We describe an ensemble neural language generator, and present several novel methods for data representation and augmentation that yield improved results in our model. We test the model on three datasets in the restaurant, TV and laptop domains, and report both objective and subjective evaluations of our best model. Using a range of automatic metrics, as well as human evaluators, we show that our approach achieves better results than state-of-the-art models on the same datasets.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Semantic Noise Matters for Neural Natural Language Generation (2019, methodology): “We plan to further improve our slot matching script based on errors found during the manual evaluation and comparison to Juraska et al. (2018).”
Neural Generation for Czech: Data and Baselines (2019, methodology): “…example is the delexicalization technique used by most current generators (e.g., Oh and Rudnicky, 2000; Mairesse et al., 2010; Wen et al., 2015a,b; Juraska et al., 2018): It is generally assumed that attribute (slot) values from the input meaning representation (MR) can be replaced by…”
Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge (2019, methodology): “SHEFF1, FORGE1 and SLUG-ALT produce higher-level sentences more frequently and thus show among the most distant from other systems.”
Findings of the E2E NLG Challenge (2018, methodology): “We consider the S LUG system (Juraska et al., 2018), a seq2seq-based ensemble system with a reranker, as the overall winner of the E2E NLG challenge.”
Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue (2019, methodology): “(Dušek and Jurcıcek, 2016) and the challenge winner SLUG (Juraska et al., 2018) on BLEU (0.”
End-to-End Content and Plan Selection for Data-to-Text Generation (2018, methodology): “While Juraska et al. (2018) adds an explicit penalty for each attribute that is not part of a generated text, we aim to implicitly reduce this number with the coverage penalty.”
Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation (2020, methodology): “More recent methods use Seq2Seq architecture (Sutskever et al., 2014) to combine planning and realization into an end-to-end network and have achieved the state-of-the-art on a variety of generation tasks (Lebret et al., 2016; Trisedya et al., 2018; Juraska et al., 2018; Reed et al., 2018).”
Handling Rare Items in Data-to-Text Generation (2018, background): “Among them, six teams applied delexi-calisation (Chen et al., 2018; Davoodi et al., 2018; Juraska et al., 2018; Puzikov and Gurevych, 2018; Trisedya et al., 2018; van der Lee et al., 2017), three resorted to the copy mechanism (Chen, 2018; Elder et al., 2018; Gehrmann et al., 2018), two developed…”
Slot-consistent NLG for Task-oriented Dialogue Systems with Iterative Rectification Network (2020, background): “However, this step can introduce missing and misplaced slots, because of modeling errors or unaligned training data (Balakrishnan et al., 2019; Nie et al., 2019; Juraska et al., 2018).”
Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020, methodology): “Results on E2E On the E2E dataset, apart from our implementations, we also compare agianst outputs from the SLUG (Juraska et al., 2018), the overall winner of the E2E challenge (seq2seq-based), DANGNT (Nguyen and Tran, 2018), the best grammar rule based model, TUDA (Puzikov and Gurevych, 2018), the ”
Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity (2020, methodology): “For this, we use the heuristics by Shimorina and Gar-dent (2018) for WebNLG, by Juraska et al. (2018) for ViGGO, and by Duˇsek et al. (2019) for Cleaned E2E.”
Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference (2020, unknown): “• Slug2Slug slot aligner (Juraska et al., 2018) is based on keyword matches.”
Iterative Delexicalization for Improved Spoken Language Understanding (2019, background): “Using delexicalization to improve performance of other NLP systems has also been studied in the context of natural language generation [11], dependency parsing [12, 13], semantic parsing [14], and representation learning [10].”
Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models (2019, methodology): “…results obtained with this method on the E2E challenge benchmark, show that the method can achieve a similar score of the winner of the challenge (Juraska et al., 2018) but with far less annotated data and without using any pre-processing (delexicalisation, data augmentation) or re-scoring…”
AggGen: Ordering and Aggregating while Generating (2021, background): “The prevailing sequence-to-sequence (seq2seq) architectures typically address this issue via reranking (Wen et al., 2015a; Dušek and Jurčı́ček, 2016; Juraska et al., 2018) or some sophisticated training techniques (Nie et al.”
Controlling hallucinations at word level in data-to-text generation (2021, unknown): “2019) or hand crafted rules (Juraska et al. 2018) can help to reduce hallucinations.”
A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation (2019, methodology): “…than 50% relative unaligned noise from original MR-text pairs, and a vanilla sequence-to-sequence model trained on the reﬁned data can improve content correctness in both human and automatic evaluations, when compared with the current state-of-the-art neural ensemble system (Juraska et al., 2018).”
Neural Data-to-Text Generation with LM-based Text Augmentation (2021, result): “SLUG uses a heuristic slot aligner based on a set of handcrafted rules and combines a complex pipeline of data augmentation, selection, model ensemble and reranker.”