Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue

By Anusha Balakrishnan, Jinfeng Rao, Kartikeya Upasani, Michael. White, Rajen Subba. 2019

87 citations recorded by the source corpus.

Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems. Avenues like the E2E NLG Challenge have encouraged the development of neural approaches, particularly sequence-to-sequence (Seq2Seq) models for this problem. The semantic representations used, however, are often underspecified, which places a higher burden on the generation model for sentence planning, and also limits the extent to which generated responses can be controlled in a live system. In this paper, we (1) propose using tree-structured semantic representations, like those used in traditional rule-based NLG systems, for better discourse-level structuring and sentence-level planning; (2) introduce a challenging dataset using this representation for the weather domain; (3) introduce a constrained decoding approach for Seq2Seq models that leverages this representation to improve semantic correctness; and (4) demonstrate promising results on our dataset and the E2E dataset.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Structure-to-Text Generation with Self-Training, Acceptability Classifiers and Context-Conditioning for the GEM Shared Task (2021, background): “However, training reliable and low-latency generative models has typically required tens of thousands of training samples (Balakrishnan et al., 2019; Novikova et al., 2017).”
Autoregressive Entity Retrieval (2020, background): “…role labelling (Daza & Frank, 2018), discourse representation structure parsing (Liu et al., 2018), generation of ﬂuent natural language responses from structured semantic representations (Balakrishnan et al., 2019), generation and parsing of abstract meaning representation (Konstas et al., 2017).”
Survey of Hallucination in Natural Language Generation (2022, methodology): “[6] propose to adopt tree-structured semantic representations and add constraints on decoding, Li et al.”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “Constrained beam search [6] was also proposed to generate desirable responses by constraining a generated response to obey the input structure.”
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses (2021, methodology): “(2014) and Balakrishnan et al. (2019) use parse trees among other techniques to enforce constraints in the generated text. Hokamp and Liu (2017); Post and Vilar (2018) propose variants of Grid Beam Search, which generate output that include lexical constraints. Miao et al. (2019); Zhang et al. (2020”
Getting to Production with Few-shot Natural Language Generation Models (2021, methodology): “In the full data setting, retaining the tree structure helps with more accurate natural language generation (Table 4), which is in line with observations in Balakrishnan et al. (2019).”
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding (2021, background): “with the issue, but are often custom-made for taskoriented dialogue (Balakrishnan et al., 2019).”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Balakrishnan et al. (2019) used linearized tree-structured representation as input to obtain control over discourse-level and sentence-level semantic concepts. Kale and Rastogi (2020) used templates to improve the semantic correctness of generated responses.”
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022, background): “For example, the challenge set for the conversational weather dataset (Balakrishnan et al., 2019) selects examples from the original test split with complex discourse relations.”
Structural Information Preserving for Graph-to-Text Generation (2020, background): “This is a very important issue, because of its wide existence across many neural graph-to-text generation models, hindering the usability of these models for real-world applications (Duˇsek et al., 2018, 2019; Balakrishnan et al., 2019).”
Slot-consistent NLG for Task-oriented Dialogue Systems with Iterative Rectification Network (2020, methodology): “Although neural generators can be trained end-to-end, they suffer from hallucination phenomenon (Balakrishnan et al., 2019).”
Neural NLG for Methodius: From RST Meaning Representations to Texts (2020, methodology): “Recently, Balakrishnan et al. (2019a) have argued that discourse relations should be reintroduced into neural generation in order to enable the correct expression of these relations to be more reliably controlled.”
Improved Natural Language Generation via Loss Truncation (2020, methodology): “Perhaps surprisingly, improving BLEU score to 0.72 almost closes the gap to using complex tree-structured semantic representations, which achieves a BLEU score of 0.74 (Balakrishnan et al., 2019).”
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints (2020, background): “In addition, Balakrishnan et al. (2019) leverage constrained decoding to improve semantic correctness for response generation.”
Improving Text-to-Text Pre-trained Models for the Graph-to-Text Task (2020, background): “There are also works on adding semantic supervi-sions (Reed et al., 2018; Balakrishnan et al., 2019), however, it requires extensive efforts to annotate discourse relations in each reference text.”
Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020, methodology): “For example, we can perform a similar constrained decoding as in Balakrishnan et al. (2019) to rule out outputs with undesired patterns.”
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2021, background): “, 2020b), and template/scaffold guided schema with explicit plans (Ma et al., 2019; Moryossef et al., 2019; Balakrishnan et al., 2019; Du et al., 2020; Liu et al., 2021), e.”